Skip to main content
SpringerLink
Log in

Enhancing Expression Recognition in the Wild with Unlabeled Reference Data

  • Conference paper
Computer Vision – ACCV 2012 (ACCV 2012)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 7725))

Included in the following conference series:

Abstract

Facial expression recognition is an important task in human-computer interaction. Some methods work well on ”lab-controlled” data. However, their performances degenerate dramatically on real-world data as expression covers large variations, including pose, illumination, occlusion, and even culture change. To deal with this problem, large scale data is definitely needed. On the other hand, collecting and labeling wild expression data can be difficult and time consuming. In this paper, aiming at robust expression recognition in wild which suffers from the mentioned problems, we propose a semi-supervised method to make use of the large scale unlabeled data in two steps: 1) We enrich reference manifolds using selected unlabeled data which are closed to certain kind of expression. The learned manifolds can help smooth the variation of original data and provide reliable metric to maintain semantic similarity of expression; 2) To elevate the original labeled set for enhanced training, we iteratively employ the semi-supervised clustering to assign labels for unlabeled data and add the most discriminant ones into the labeled set. Experiments on the latest wild expression database SFEW and GENKI show that the proposed method can effectively exploit unlabeled data to improve the performance on real-world expression recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Pantic, M., Rothkrantz, L.: Automatic analysis of facial expressions: The state of the art. IEEE T PAMI 22, 1424–1445 (2000)

    Article  Google Scholar 

  2. Zeng, Z., Pantic, M., Roisman, G., Huang, T.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE T PAMI 31, 39–58 (2009)

    Article  Google Scholar 

  3. Shan, C., Gong, S., McOwan, P.: Facial expression recognition based on local binary patterns: A comprehensive study. IVC 27, 803–816 (2009)

    Article  Google Scholar 

  4. Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using phog and lpq features. In: FG, pp. 878–883 (2011)

    Google Scholar 

  5. Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.: Learning active facial patches for expression analysis. In: CVPR, pp. 2562–2569 (2012)

    Google Scholar 

  6. Chew, S., Lucey, S., Lucey, P., Sridharan, S.: Improved facial expression recognition via uni-hyperplane classification. In: CVPR, pp. 2554–2561 (2012)

    Google Scholar 

  7. Kumar, N., Berg, A., Belhumeur, P., Nayar, S.: Attribute and simile classifiers for face verification. In: ICCV, pp. 365–372 (2009)

    Google Scholar 

  8. Yin, Q., Tang, X., Sun, J.: An associate-predict model for face recognition. In: CVPR, pp. 497–504 (2011)

    Google Scholar 

  9. Schroff, F., Treibitz, T., Kriegman, D., Belongie, S.: Pose, illumination and expression invariant pairwise face-similarity measure via doppelgänger list comparison. In: ICCV, pp. 2494–2501 (2011)

    Google Scholar 

  10. Su, Y., Shan, S., Chen, X., Gao, W.: Adaptive generic learning for face recognition from a single sample per person. In: CVPR, pp. 2699–2706 (2010)

    Google Scholar 

  11. Cohen, I., Sebe, N., Gozman, F., Cirelo, M., Huang, T.: Learning bayesian network classifiers for facial expression recognition both labeled and unlabeled data. In: CVPR, vol. 1, pp. 595–601 (2003)

    Google Scholar 

  12. Cohen, I., Sebe, N., Cozman, F., Huang, T.: Semi-supervised learning for facial expression recognition. In: ACM SIGMM Workshop on MIR, pp. 17–22 (2003)

    Google Scholar 

  13. Abdel-hady, M., Schels, M., Schwenker, F., Palm, G.: Semi-supervised facial expressions annotation using co-training with fast probabilistic tri-class svms. ANN, 70–75 (2010)

    Google Scholar 

  14. Saul, L., Roweis, S.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. JMLR 4, 119–155 (2003)

    MathSciNet  Google Scholar 

  15. Chang, Y., Hu, C., Turk, M.: Manifold of facial expression. In: AMFG, pp. 28–35 (2003)

    Google Scholar 

  16. Shan, C., Gong, S., McOwan, P.: Appearance manifold of facial expression. CVHCI, 221–230 (2005)

    Google Scholar 

  17. Xiao, R., Zhao, Q., Zhang, D., Shi, P.: Facial expression recognition on multiple manifolds. PR 44, 107–116 (2011)

    Article  MATH  Google Scholar 

  18. He, X., Niyogi, P.: Locality preserving projections. NIPS 16, 153–160 (2004)

    Google Scholar 

  19. Tenenbaum, J., De Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)

    Article  Google Scholar 

  20. Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. NIPS 14, 585–591 (2001)

    Google Scholar 

  21. Chen, Y., Rege, M., Dong, M., Hua, J.: Non-negative matrix factorization for semi-supervised data clustering. KIS 17, 355–379 (2008)

    Google Scholar 

  22. Kulis, B., Basu, S., Dhillon, I., Mooney, R.: Semi-supervised graph clustering: a kernel approach. In: ICML, pp. 457–464 (2005)

    Google Scholar 

  23. Dhall, A., Goecke, R., Lucey, S., Gedeon, T.: Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. In: ICCV Workshop on BeFIT, pp. 2106–2112 (2011)

    Google Scholar 

  24. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR, vol. 1, pp. 511–518 (2001)

    Google Scholar 

  25. Chang, C., Lin, C.: Libsvm: a library for support vector machines. ACM TIST 2,  27 (2011)

    Google Scholar 

  26. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)

    Google Scholar 

  27. Huang, G., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Technical Report07-49, University of Massachusetts, Amherst (2008)

    Google Scholar 

  28. GENKI-4K, http://mplab.ucsd.edu , the mplab genki dataset: Genki-4k subset

  29. Whitehill, J., Littlewort, G., Fasel, I., Bartlett, M., Movellan, J.: Toward practical smile detection. IEEE T PAMI 31, 2106–2111 (2009)

    Article  Google Scholar 

  30. Shan, C.: Smile detection by boosting pixel differences. IEEE T IP 21, 431–436 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Lab of Intelligent Information Processing, Chinese Academy of Sciences (CAS), Beijing, 100190, China

    Mengyi Liu, Shaoxin Li, Shiguang Shan & Xilin Chen

  2. Institute of Computing Technology, CAS, Beijing, 100190, China

    Mengyi Liu, Shaoxin Li, Shiguang Shan & Xilin Chen

  3. Graduate University of Chinese Academy of Sciences, Beijing, 100049, China

    Mengyi Liu & Shaoxin Li

Authors
  1. Mengyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaoxin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiguang Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xilin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, 151-744, Seoul, Korea

    Kyoung Mu Lee

  2. Microsoft Research Asia, No. 5, Danling st., Haidian district, 100080, Beijing, P.R. China

    Yasuyuki Matsushita

  3. School of Interactive Computing, Georgia Institute of Technology, 801 Atlantic Drive, CCB 315, 30332, Atlanta, GA, USA

    James M. Rehg

  4. Institute of Automation, National Laboratory of Pattern Recognition, Chinese Academy of Sciences, Zhong Quan Cun East Road 95, Haidian District, 100 190, Beijing, P.R. China

    Zhanyi Hu

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Liu, M., Li, S., Shan, S., Chen, X. (2013). Enhancing Expression Recognition in the Wild with Unlabeled Reference Data. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds) Computer Vision – ACCV 2012. ACCV 2012. Lecture Notes in Computer Science, vol 7725. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37444-9_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37444-9_45

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37443-2

  • Online ISBN: 978-3-642-37444-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation