Skip to main content
SpringerLink
Log in

Visual and Thermal Data for Pedestrian and Cyclist Detection

  • Conference paper
  • First Online:
Towards Autonomous Robotic Systems (TAROS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11650))

Included in the following conference series:

Abstract

With the continued advancement of autonomous vehicles and their implementation in public roads, accurate detection of vulnerable road users (VRUs) is vital for ensuring safety. To provide higher levels of safety for these VRUs, an effective detection system should be employed that can correctly identify VRUs in all types of environments (e.g. VRU appearance, crowded scenes) and conditions (e.g. fog, rain, night-time). This paper presents optimal methods of sensor fusion for pedestrian and cyclist detection using Deep Neural Networks (DNNs) for higher levels of feature abstraction. Typically, visible sensors have been utilized for this purpose. Recently, thermal sensors system or combination of visual and thermal sensors have been employed for pedestrian detection with advanced detection algorithm. DNNs have provided promising results for improving the accuracy of pedestrian and cyclist detection. This is because they are able to extract features at higher levels than typical hand-crafted detectors. Previous studies have shown that amongst the several sensor fusion techniques that exist, Halfway Fusion has provided the best results in terms of accuracy and robustness. Although sensor fusion and DNN implementation have been used for pedestrian detection, there is considerably less research undertaken for cyclist detection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Angelova, A., Krizhevsky, A., Vanhoucke, V., Ogale, A., Ferguson, D.: Real-time pedestrian detection with deep network cascades. In: Proceedings of the British Machine Vision Conference, pp. 1–12 (2015). https://doi.org/10.5244/C.29.32

  2. Baek, J., Hong, S., Kim, J., Kim, E.: Efficient pedestrian detection at nighttime using a thermal camera. Sens. (Switz.) 17(8), 1850 (2017). https://doi.org/10.3390/s17081850

    Article  Google Scholar 

  3. Bertozzi, M., Broggi, A., Caraffi, C., Del Rose, M., Felisa, M., Vezzoni, G.: Pedestrian detection by means of far-infrared stereo vision. Comput. Vis. Image Underst. 106(2–3), 194–204 (2007). https://doi.org/10.1016/j.cviu.2006.07.016

    Article  Google Scholar 

  4. Biswas, S.K., Milanfar, P.: Linear support tensor machine with LSK channels: pedestrian detection in thermal infrared images. IEEE Trans. Image Proc. 26(9), 4229–4242 (2017). https://doi.org/10.1109/TIP.2017.2705426

    Article  MathSciNet  MATH  Google Scholar 

  5. Chang, S.L., Yang, F.T., Wu, W.P., Cho, Y.A., Chen, S.W.: Nighttime pedestrian detection using thermal imaging based on HOG feature. In: Proceedings 2011 International Conference on System Science and Engineering, pp. 694–698. IEEE (2011). https://doi.org/10.1109/ICSSE.2011.5961992

  6. Dai, C., Zheng, Y., Li, X.: Layered representation for pedestrian detection and tracking in infrared imagery. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005) - Workshops. vol. 3, pp. 13–13. IEEE (2005). https://doi.org/10.1109/CVPR.2005.483

  7. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893. IEEE (2005). https://doi.org/10.1109/CVPR.2005.177

  8. Davis, J., Sharma, V.: Robust detection of people in thermal imagery. In: 2004 Proceedings of the 17th International Conference on Pattern Recognition, ICPR, pp. 713–716. IEEE (2004). https://doi.org/10.1109/ICPR.2004.1333872

  9. European Road Safety Observatory: Traffic Safety Basic Facts 2012. Technical report, European Road Safety Observatory (2012)

    Google Scholar 

  10. Gandhi, T., Trivedi, M.M.: Pedestrian protection systems: issues, survey, and challenges. IEEE Trans. Intell. Transp. Syst. 8(3), 413–430 (2007). https://doi.org/10.1109/TITS.2007.903444

    Article  Google Scholar 

  11. Gerónimo, D., López, A.M., Sappa, A.D., Graf, T.: Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1239–1258 (2010). https://doi.org/10.1109/TPAMI.2009.122

    Article  Google Scholar 

  12. Gilmore, E.T., Frazier, P.D, Chouikha, M.F.: Improved human detection using image fusion. In: Proceedings of the IEEE ICRA (2009)

    Google Scholar 

  13. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580–587 (2014)

    Google Scholar 

  14. González, A., et al.: Pedestrian detection at day/night time with visible and FIR cameras: a comparison. Sens. (Switz.) 16(6), 1–11 (2016). https://doi.org/10.3390/s16060820

    Article  Google Scholar 

  15. Hurney, P., Jones, E., Waldron, P., Glavin, M., Morgan, F.: Night-time pedestrian classification with histograms of oriented gradients-local binary patterns vectors. IET Intell. Transp. Syst. 9(1), 75–85 (2015). https://doi.org/10.1049/iet-its.2013.0163

    Article  Google Scholar 

  16. Hwang, S., Park, J., Kim, N., Choi, Y., So, I.: Multispectral pedestrian detection: benchmark dataset and baseline. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1037–1045 (2015)

    Google Scholar 

  17. Zhao, J., Cheung, S.C.S.: Human segmentation by fusing visible-light and thermal imaginary. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 1185–1192. IEEE (2009). https://doi.org/10.1109/ICCVW.2009.5457476, http://ieeexplore.ieee.org/document/5457476/

  18. Kocic, J., Jovicic, N., Drndarevic, V.: Sensors and sensor fusion in autonomous vehicles. In: 2018 26th Telecommunications Forum (TELFOR), pp. 420–425. IEEE (2018). https://doi.org/10.1109/TELFOR.2018.8612054, https://ieeexplore.ieee.org/document/8612054/

  19. Lee, J.H., et al.: Robust pedestrian detection by combining visible and thermal infrared cameras. Sens. (Switz.) 15(5), 10580–10615 (2015). https://doi.org/10.3390/s150510580

    Article  Google Scholar 

  20. Li, W., Zheng, D., Zhao, T., Yang, M.: An effective approach to pedestrian detection in thermal imagery. In: 2012 8th International Conference on Natural Computation, pp. 325–329. IEEE (2012). https://doi.org/10.1109/ICNC.2012.6234621

  21. Li, X., et al.: A new benchmark for vision-based cyclist detection. In: Proceedings of IEEE Intelligent Vehicles Symposium, pp. 1028–1033 (2016)

    Google Scholar 

  22. Li, X., et al.: A unified framework for concurrent pedestrian and cyclist detection. IEEE Trans. Intell. Transp. Syst. 18(2), 269–281 (2017). https://doi.org/10.1109/TITS.2016.2567418

    Article  Google Scholar 

  23. Li, Z., Zhang, J., Wu, Q., Geers, G.: Feature enhancement using gradient salience on thermal image. In: 2010 International Conference on Digital Image Computing: Techniques and Applications, pp. 556–562. IEEE (2010). https://doi.org/10.1109/DICTA.2010.99, http://ieeexplore.ieee.org/document/5692620/

  24. Liu, J., Zhang, S., Wang, S., Metaxas, D.N.: Multispectral deep neural networks for pedestrian detection. In: British Machine Vision Conference, pp. 1–13 (2016)

    Google Scholar 

  25. Mikolajczyk, K., Schmid, C., Zisserman, A.: Human detection based on a probabilistic assembly of robust part detectors. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 69–82. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24670-1_6

    Chapter  Google Scholar 

  26. Neagoe, V.E., Ciotec, A.D., Bărar, A.P.: A concurrent neural network approach to pedestrian detection in thermal imagery. In: 2012 9th International Conference on Communications (COMM), pp. 133–136 (2012). https://doi.org/10.1109/ICComm.2012.6262539

  27. Olmeda, D., Armingol, J.M., de la Escalera, A.: Discrete features for rapid pedestrian detection in infrared images. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3067–3072. IEEE (2012). https://doi.org/10.1109/IROS.2012.6385928, http://ieeexplore.ieee.org/document/6385928/

  28. O’Malley, R., Jones, E., Glavin, M.: Detection of pedestrians in far-infrared automotive night vision using region-growing and clothing distortion compensation. Infrared Phys. Technol. 53(6), 439–449 (2010). https://doi.org/10.1016/J.INFRARED.2010.09.006

    Article  Google Scholar 

  29. Tian, W., Lauer, M.: Fast and robust cyclist detection for monocular camera systems. In: International joint Conference on Computer Vision Imaging and Computer Graphics Theory and Applications (VISIGRAPP) (2015)

    Google Scholar 

  30. Tian, W., Lauer, M.: Detection and orientation estimation for cyclists by max pooled features. In: Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, SCITEPRESS - Science and Technology Publications, pp. 17–26 (2017). https://doi.org/10.5220/0006085500170026

  31. Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T., Smeulders, A.W.M.: Selective search for object recognition. Int. J. Comput. Vision 104(2), 154–171 (2013). https://doi.org/10.1007/s11263-013-0620-5

    Article  Google Scholar 

  32. Viola, P., Jones, M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Proceedings of the 9th IEEE International Conference on Computer Vision, vol. 1, no. 9, pp. 734–741 (2003). https://doi.org/10.1109/ICCV.2003.1238422

  33. Wagner, J., Fischer, V., Herman, M.: Multispectral pedestrian detection using deep fusion convolutional neural networks. In: European Symposium on Artificial Neural Networks (2016)

    Google Scholar 

  34. World Health Organisation: Global Status Report on Road Safety 2015 - Summary (2015)

    Google Scholar 

  35. Wu, T.E., Tsai, C.C., Guo, J.I.: LiDAR/camera sensor fusion technology for pedestrian detection. In: 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1675–1678. IEEE (2017). https://doi.org/10.1109/APSIPA.2017.8282301

  36. Xu, D., Ouyang, W., Ricci, E., Wang, X., Sebe, N.: Learning cross-modal deep representations for robust pedestrian detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4236–4244. IEEE (2017). https://doi.org/10.1109/CVPR.2017.451, http://ieeexplore.ieee.org/document/8099934/

  37. Xu, F., Liu, X., Fujimura, K.: Pedestrian detection and tracking with night vision. IEEE Trans. Intell. Transp. Syst. 6(1), 63–71 (2005). https://doi.org/10.1109/TITS.2004.838222

    Article  Google Scholar 

  38. Chen, Y., Han, C.: Night-time pedestrian detection by visual-infrared video fusion. In: 2008 7th World Congress on Intelligent Control and Automation, pp. 5079–5084. IEEE (2008). https://doi.org/10.1109/WCICA.2008.4593753

  39. Zhang, S., Benenson, R., Omran, M., Hosang, J., Schiele, B.: How far are we from solving pedestrian detection?. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1259–1267 (2016). https://doi.org/10.1109/CVPR.2016.141

  40. Zhao, X., He, Z., Zhang, S., Liang, D.: Robust pedestrian detection in thermal infrared imagery using a shape distribution histogram feature and modified sparse representation classification. Pattern Recogn. 48(6), 1947–1960 (2015). https://doi.org/10.1016/J.PATCOG.2014.12.013

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, Electronics and Mathematics, Coventry University, Coventry, CV1 5FB, UK

    Sarfraz Ahmed, M. Nazmul Huda, Sujan Rajbhandari, Chitta Saha & Mark Elshaw

  2. School of Mechanical, Aerospace and Automotive Engineering, Coventry University, Coventry, CV1 5FB, UK

    Stratis Kanarachos

Authors
  1. Sarfraz Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Nazmul Huda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sujan Rajbhandari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chitta Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark Elshaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stratis Kanarachos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarfraz Ahmed .

Editor information

Editors and Affiliations

  1. Queen Mary University of London, London, UK

    Kaspar Althoefer

  2. Queen Mary University of London, London, UK

    Jelizaveta Konstantinova

  3. Queen Mary University of London, London, UK

    Ketao Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ahmed, S., Huda, M.N., Rajbhandari, S., Saha, C., Elshaw, M., Kanarachos, S. (2019). Visual and Thermal Data for Pedestrian and Cyclist Detection. In: Althoefer, K., Konstantinova, J., Zhang, K. (eds) Towards Autonomous Robotic Systems. TAROS 2019. Lecture Notes in Computer Science(), vol 11650. Springer, Cham. https://doi.org/10.1007/978-3-030-25332-5_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-25332-5_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-25331-8

  • Online ISBN: 978-3-030-25332-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation