Abstract
In this paper, we introduce an efficient framework to subtract the background from both visible and thermal imagery for pedestrians’ detection in the urban scene. We use a deep neural network (DNN) to train the background subtraction model. For the training of the DNN, we first generate an initial background map and then employ randomly 5% video frames, background map, and manually segmented ground truth. Then we apply a cognition-based post-processing to further smooth the foreground detection result. We evaluate our method against our previous work and 11 recently widely cited method on three challenge video series selected from a publicly available color-thermal benchmark dataset OCTBVS. Promising results have been shown that the proposed DNN-based approach can successfully detect the pedestrians with good shape in most scenes regardless of illuminate changes and occlusion problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Babaee, M., Dinh, D.T., Rigoll, G.: A deep convolutional neural network for background subtraction. arXiv preprint arXiv:170201731 (2017)
Barnich, O., Van Droogenbroeck, M.: ViBe: a powerful random technique to estimate the background in video sequences. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2009, pp. 945–948. IEEE (2009)
Bloisi, D., Iocchi, L.: Independent multimodal background subtraction. In: CompIMAGE, pp. 39–44 (2012)
Chai, Y., Ren, J., Zhao, H., Li, Y., Ren, J., Murray, P.: Hierarchical and multi-featured fusion for effective gait recognition under variable scenarios. Pattern Anal. Appl. 19, 905–917 (2016)
Davis, J.W., Keck, M.A.: A two-stage template approach to person detection in thermal imagery. In: Null, pp. 364–369. IEEE (2005)
Godbehere, A.B., Matsukawa, A., Goldberg, K.: Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation. In: American Control Conference (ACC), pp. 4305–4312. IEEE (2012)
Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., Wu, F.: Background prior-based salient object detection via deep reconstruction residual. IEEE Trans. Circ. Syst. Video Technol. 25, 1309–1321 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 38–43. IEEE (2012)
Kim, D.-E., Kwon, D.-S.: Pedestrian detection and tracking in thermal images using shape features. In: 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pp. 22–25. IEEE (2015)
Maddalena, L., Petrosino, A.: A fuzzy spatial coherence-based approach to background/foreground separation for moving object detection. Neural Comput. Appl. 19, 179–186 (2010)
Nguyen, T.P., Pham, C.C., Ha, S.V.-U., Jeon, J.W.: Change detection by training a triplet network for motion feature extraction. IEEE Trans. Circ. Syst. Video Technol. (2018, in press)
Noh, S., Jeon, M.: A new framework for background subtraction using multiple cues. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7726, pp. 493–506. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37431-9_38
Ren, J., Han, J., Dalla Mura, M.: Special issue on multimodal data fusion for multidimensional signal processing. Multidimension. Syst. Signal Process. 27, 801–805 (2016)
Ren, J., Jiang, J., Wang, D., Ipson, S.: Fusion of intensity and inter-component chromatic difference for effective and robust colour edge detection. IET Image Process. 4, 294–301 (2010)
St-Charles, P.-L., Bilodeau, G.-A.: Improving background subtraction using local binary similarity patterns. In: 2014 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 509–515. IEEE (2014)
St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R.: Flexible background subtraction with self-balanced local sensitivity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 408–413 (2014)
Vacavant, A., Chateau, T., Wilhelm, A., Lequievre, L.: A benchmark dataset for foreground/background extraction. In: ACCV 2012 Workshop: Background Models Challenge (2012)
Wang, Y., Jodoin, P.-M., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: CDnet 2014: an expanded change detection benchmark dataset. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 393–400. IEEE (2014)
Wang, Z., Ren, J., Zhang, D., Sun, M., Jiang, J.: A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos. Neurocomputing 287, 68–83 (2018)
Yan, Y., Ren, J., Li, Y., Windmill, J., Ijomah, W.: Fusion of dominant colour and spatial layout features for effective image retrieval of coloured logos and trademarks. In: 2015 IEEE International Conference on Multimedia Big Data (BigMM), pp. 306–311. IEEE (2015)
Yan, Y., Ren, J., Li, Y., Windmill, J.F., Ijomah, W., Chao, K.-M.: Adaptive fusion of color and spatial features for noise-robust retrieval of colored logo and trademark images. Multidimension. Syst. Signal Process. 27, 945–968 (2016)
Yan, Y., et al.: Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement. Pattern Recogn. 79, 65–78 (2018)
Yan, Y., et al.: Cognitive fusion of thermal and visible imagery for effective detection and tracking of pedestrians in videos. Cognit. Comput. 10, 94–104 (2018)
Zabalza, J., et al.: Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing 185, 1–10 (2016)
Zhao, Z., Bouwmans, T., Zhang, X., Fang, Y.: A fuzzy background modeling approach for motion detection in dynamic backgrounds. In: Wang, F.L., Lei, J., Lau, R.W.H., Zhang, J. (eds.) CMSP 2012. CCIS, vol. 346, pp. 177–185. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35286-7_23
Zheng, J., Liu, Y., Ren, J., Zhu, T., Yan, Y., Yang, H.: Fusion of block and keypoints based approaches for effective copy-move image forgery detection. Multidimension. Syst. Signal Process. 27, 989–1005 (2016)
Zhou, X., Yang, C., Yu, W.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35, 597–610 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Yan, Y., Zhao, H., Kao, FJ., Vargas, V.M., Zhao, S., Ren, J. (2018). Deep Background Subtraction of Thermal and Visible Imagery for Pedestrian Detection in Videos. In: Ren, J., et al. Advances in Brain Inspired Cognitive Systems. BICS 2018. Lecture Notes in Computer Science(), vol 10989. Springer, Cham. https://doi.org/10.1007/978-3-030-00563-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-00563-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00562-7
Online ISBN: 978-3-030-00563-4
eBook Packages: Computer ScienceComputer Science (R0)