Abstract
In this paper, a spontaneous facial expression recognition method using infrared thermal videos is proposed. Firstly, the sequence features are extracted from the infrared thermal horizontal and vertical temperature difference sequences of different facial sub-regions. Secondly, a feature subset is selected according to their F-values. Thirdly, the Adaboost algorithm, with the weak classifiers of k-Nearest Neighbor, is used to classify facial expressions in arousal and valence dimensions. Finally, experiments on the Natural Visible and Infrared facial Expression (USTC-NVIE) database demonstrates the effectiveness of our approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1), 39–58 (2009)
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognition 36(1), 259–275 (2003)
Khan, M.M., Ward, R.D., Ingleby, M.: Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature. ACM Transactions on Applied Perception (TAP) 6(1), 1–22 (2009)
Jenkins, S., Brown, R., Rutterford, N.: Comparing thermographic, eeg, and subjective measures of affective experience during simulated product interactions. International Journal of Design 3(2), 53–65 (2009)
Levine, J.A., Pavlidis, I., Cooper, M.: The face of fear. The Lancet 357(9270), 1757 (2001)
Nozawa, A., Tacano, M.: Correlation analysis on alpha attenuation and nasal skin temperature. Journal of Statistical Mechanics: Theory and Experiment, P01007 (2009)
Nakanishi, R., Imai-Matsumura, K.: Facial skin temperature decreases in infants with joyful expression. Infant Behavior and Development 31(1), 137–144 (2008)
Pavlidis, I., Levine, J.: Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine 21(6), 56–64 (2002)
Puri, C., Olson, L., Pavlidis, I., Levine, J., Starren, J.: Stresscam: non-contact measurement of users’ emotional states through thermal imaging. In: Extended Abstracts on Human Factors in Computing Systems, CHI 2005, pp. 1725–1728. ACM (2005)
Khan, M.M., Ward, R.D., Ingleby, M.: Infrared thermal sensing of positive and negative affective states. In: IEEE Conference on Robotics, Automation and Mechatronics, pp. 1–6. IEEE (2006)
Nhan, B.R., Chau, T.: Classifying affective states using thermal infrared imaging of the human face. IEEE Transactions on Biomedical Engineering 57(4), 979–987 (2010)
Jarlier, S., Grandjean, D., Delplanque, S., N’Diaye, K., Cayeux, I., Velazco, M., Sander, D., Vuilleumier, P., Scherer, K.: Thermal analysis of facial muscles contractions. IEEE Transactions on Affective Computing (99), 1 (2011)
Yoshitomi, Y.: Facial expression recognition for speaker using thermal image processing and speech recognition system. In: Proceedings of the 10th WSEAS International Conference on Applied Computer Science, pp. 182–186. World Scientific and Engineering Academy and Society, WSEAS (2010)
Krzywicki, A.T., He, G., O’Kane, B.L.: Analysis of facial thermal variations in response to emotion: eliciting film clips. Proceedings of SPIE 7343, 734312 (2009)
Koda, Y., Yoshitomi, Y., Nakano, M., Tabuse, M.: A facial expression recognition for a speaker of a phoneme of vowel using thermal image processing and a speech recognition system. In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2009, pp. 955–960. IEEE (2009)
Hernández, B., Olague, G., Hammoud, R., Trujillo, L., Romero, E.: Visual learning of texture descriptors for facial expression recognition in thermal imagery. Computer Vision and Image Understanding 106(2-3), 258–269 (2007)
Khan, M.M., Ingleby, M., Ward, R.D.: Automated facial expression classification and affect interpretation using infrared measurement of facial skin temperature variations. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 1(1), 91–113 (2006)
Trujillo, L., Olague, G., Hammoud, R., Hernandez, B.: Automatic feature localization in thermal images for facial expression recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, CVPR Workshops, p. 14. IEEE (2005)
Wang, P., Ji, Q.: Multi-view face and eye detection using discriminant features. Computer Vision and Image Understanding 105(2), 99–111 (2007)
Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech and Signal Processing 29(6), 1153–1160 (1981)
Wang, Y., Ai, H., Wu, B., Huang, C.: Real time facial expression recognition with adaboost. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 3, pp. 926–929. IEEE (2004)
Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573. IEEE (2005)
Wang, S., Liu, Z., Lv, S., Lv, Y., Wu, G., Peng, P., Chen, F., Wang, X.: A natural visible and infrared facial expression database for expression recognition and emotion inference. IEEE Transactions on Multimedia 12(7), 682–691 (2010)
Khan, M.M., Ward, R.D., Ingleby, M.: Automated classification and recognition of facial expressions using infrared thermal imaging. In: IEEE Conference on Cybernetics and Intelligent Systems, vol. 1, pp. 202–206. IEEE (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shen, P., Wang, S., Liu, Z. (2013). Facial Expression Recognition from Infrared Thermal Videos. In: Lee, S., Cho, H., Yoon, KJ., Lee, J. (eds) Intelligent Autonomous Systems 12. Advances in Intelligent Systems and Computing, vol 194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33932-5_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-33932-5_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33931-8
Online ISBN: 978-3-642-33932-5
eBook Packages: EngineeringEngineering (R0)