Abstract
Efficient emotional state analyzing will enable machines to understand human better and facilitate the development of applications which involve human–machine interaction. Recently, deep learning methods become popular due to their generalization ability, but the disadvantage of complicated computation could not meet the requirements of real-time characteristics. This paper proposes an emotion recognition framework based on convolution neural network, which contains less number of parameters comparatively. In order to verify the proposed framework, we train a network on a large number of facial expression images and then use the pretrained model to predict image frame taken from a single camera. The experiment shows that compared to VGG13, our network reduces the parameters by 147 times.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Suwa, M., Sugie, N., Fujimora, K.: A preliminary note on pattern recognition of human emotional expression. In: Proceedings of the 4th International Joint Conference on Pattern Recognition 1978, IAPR, pp. 408–410, Kyoto, Japan (1978)
Tian, Y.I., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2002)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: 8th IEEE International Conference on Automatic Face & Gesture Recognition 2008, pp. 1–6. Amsterdam, Netherlands (2008)
Corneanu, C.A., Simon, M.O., Cohn, J.F., et al.: Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: history, trends, and affect-related applications. IEEE Trans. Pattern Anal. Mach. Intell. 38(8), 1548–1568 (2016)
Ji, Q.: Looney: A probabilistic framework for modeling and real-time monitoring human fatigue. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 36(5), 862–875 (2006)
Ashraf, A.B., Lucey, S., Cohn, J.F.: The painful face—pain expression recognition using active appearance models. Image Vis. Comput. 27(12), 1788–1796 (2009)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conference Computer Vision Pattern Recognition 2001, vol. 1, pp. I–511 (2001)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005, CVPR, pp. 886–893, San Diego, USA (2005)
Osadchy, M., Miller, M., Lecun, Y.: Synergistic face detection and pose estimation. J. Mach. Learn. Res. 8(1), 1197–1215 (2006)
Cootes, T.F., Taylor, C.J., Cooper, D.H., et al.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–686 (2001)
Tian, Y.L., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)
Lemaire, P., Ardabilian, M., Chen, L., et al.: Fully automatic 3D facial expression recognition using differential mean curvature maps and histograms of oriented gradients. In: 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition 2013, (FG), pp. 1–7, Shanghai, China (2013)
Dapogny, A., Bailly, K., Dubuisson, S.: Dynamic facial expression recognition by joint static and multi-time gap transition classification. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition 2015, (FG), pp. 1–6, Ljubljana, Slovenia (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Proceedings of Computer Vision—ECCV 2016, vol. 9908, pp. 770–778. Springer, Cham (2016)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. JMLR, pp. 448–456 (2015)
Su, W., Boyd, S., Candes, E.J.: A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights. Adv. Neural Inf. Process. Syst. 3(1), 2510–2518 (2015)
FER2013 Dataset. https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge. Accessed 25 Jan 2019
Barsoum, E., et al.: Training deep networks for facial expression recognition with crowd-sourced label distribution. In: ACM International Conference on Multimodal Interaction ACM, pp. 279–283 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yang, H., Zhao, G., Zhang, L., Zhu, N., He, Y., Zhao, C. (2020). Real-Time Emotion Recognition Framework Based on Convolution Neural Network. In: Pan, JS., Li, J., Tsai, PW., Jain, L. (eds) Advances in Intelligent Information Hiding and Multimedia Signal Processing. Smart Innovation, Systems and Technologies, vol 157. Springer, Singapore. https://doi.org/10.1007/978-981-13-9710-3_33
Download citation
DOI: https://doi.org/10.1007/978-981-13-9710-3_33
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9709-7
Online ISBN: 978-981-13-9710-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)