Skip to main content
SpringerLink
Log in

Exaggerating Facial Expressions: A Way to Intensify Emotion or a Way to the Uncanny Valley?

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Exaggeration of facial expressions is used in animation and robotics to intensify emotions. However, modifying a human-like face can lead to an unsettling outcome. This phenomenon is known as uncanny valley. The goal of this study was to identify the realism level and magnitude of facial expression that produce the maximum amount of emotional intensity and the minimum amount of perceived strangeness. We studied the perceived intensity of emotion and perceived strangeness of faces with varying levels of realism (from schematic to photorealistic) and magnitude of facial expressions (from neutral to extremely exaggerated). We found that less realistic faces required more exaggeration to reach the emotional intensity of a real human face. While there is a range of emotional intensity that can be expressed by real human faces (from neutral to full intensity), we found that the same range of emotional intensity could be expressed by artificial faces when exaggeration was used. However, attempts to express emotional intensities outside this range using exaggeration led to strange-looking faces at all levels of realism.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ekman P, Friesen WV. The repertoire of nonverbal behavior: categories, origins, usage, and coding. Semiotica. 1969;1:49–98.

  2. Sproull L, Subramani M, Kiesler S, Walker JH, Waters K. When the interface is a face. Hum Comput Interact. 1996;11(2):97–124.

    Article  Google Scholar 

  3. Bates J. The role of emotion in believable agents. Commun ACM. 1994;37(7):122–5.

    Article  Google Scholar 

  4. Peter J, Campbell N, Melhuish C. Facial behaviour mapping: from video footage to a robot head. In: Robotics and autonomous systems; 2008. p. 1042–1049.

  5. Wu T, Butko NJ, Ruvulo P, Bartlett MS, Movellan JR. Learning to make facial expressions. In: ICDL 2009: IEEE 8th international conference on development and learning; 2009.

  6. Ishihara H, Yoshikawa Y, Asada M. Realistic child robot “Affetto” for understanding the caregiver-child attachment relationship that guides the child development. In: ICDL 2011: IEEe international conference on development and learning; Vol. 2; IEEE; 2011.

  7. Yoshikawa M, Matsumoto Y, Sumitani M, Ishiguro H. Development of an android robot for psychological support in medical and welfare fields. In: ROBIO 2011: IEEE international conference on robotics and biomimetics: IEEE; 2011.

  8. Pighin F, Hecker J, Lischinski D, Szeliski R, Salesin DH. Synthesizing realistic facial expressions from photographs. In: ACM SIGGRAPH 2006 Courses. ACM.

  9. Breazeal C, Scassellati B. How to build robots that make friends and influence people. In: IROS’99: IEEE international conference on intelligent robots and systems; Vol. 2; IEEE; 1999.

  10. van Breemen A, Yan X, Meerbeek B. iCat: an animated user-interface robot with personality. In: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems; ACM; 2005.

  11. Beskow J, McGlashan S. Olga-A conversational agent with gestures. In: Proceedings of the IJCAI’97 workshop on Animated Interface Agents-Making them Intelligent; 1997.

  12. Schiano DJ, Ehrlich SM, Rahardja K, Sheridan K. Face to interface: facial affect in (hu)man and machine. In: Proceedings of the SIGCHI conference on Human factors in computing systems; ACM; 2000.

  13. Thomas F, Johnston O. The illusion of life: disney animation. New York: Hyperion; 1995.

    Google Scholar 

  14. Lasseter J. Principles of traditional animation applied to 3D computer animation. In: ACM Siggraph Computer Graphics; ACM; 1987; 21(4).

  15. Ribeiro T, Paiva A. The illusion of robotic life: principles and practices of animation for robots. In: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction; ACM; 2012.

  16. Hyde J, Carter EJ, Kiesler S, Hodgins JK. Perceptual effects of damped and exaggerated facial motion in animated characters. In: FG 2013: IEEE international conference on automatic face and gesture recognition; IEEE; 2013.

  17. Calder AJ, Young AW, Rowland D, Perrett DI. Computer-enhanced emotion in facial expressions. Proc R Soc Lond B Biol Sci. 1997;264(1383):919–25.

    Article  CAS  Google Scholar 

  18. Benson PJ, Campbell R, Harris T, Frank MG, Tovée MJ. Enhancing images of facial expressions. Percept Psychophys. 1999;61(2):259–74.

    Article  CAS  PubMed  Google Scholar 

  19. Calder AJ, Rowland D, Young AW, Nimmo-Smith I, Keane J, Perrett DI. Caricaturing facial expressions. Cognition. 2000;76(2):105–46.

    Article  CAS  PubMed  Google Scholar 

  20. Yee N, Bailenson JN, Rickertsen K. A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces. In: Proceedings of the SIGCHI conference on Human factors in computing systems. 2007. p. 1–10.

  21. Mori M. The uncanny valley. Energy 1970;7(4): 33–35. (MacDorman KF, Minato T, Trans.).

  22. Hanson D, Olney A, Prilliman S, Mathews E, Zielke M, Hammons D, Fernandez R, Stephanou H. Upending the uncanny valley. In: Proceedings of the national conference on artificial intelligence. 20(4): 1728. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 2005.

  23. Tinwell A, Grimshaw M, Williams A. Uncanny behaviour in survival horror games. J Gaming Virtual Worlds. 2010;2(1):3–25.

    Article  Google Scholar 

  24. Geller T. Overcoming the uncanny valley. IEEE Comput Graph Appl. 2008;28(4):11–7.

    Article  PubMed  Google Scholar 

  25. Seyama J, Nagayama RS. The uncanny valley: effect of realism on the impression of artificial human faces. Presence Teleoperators Virtual Environ. 2007;16(4):337–51.

    Article  Google Scholar 

  26. Hodgkinson G. The seduction of realism. In: ACM SIGGRAPH ASIA 2009 Educators Program; ACM; 2009.

  27. MacDorman KF, Ishiguro H. The uncanny advantage of using androids in cognitive and social science research. Interact Stud. 2006;7(3):297–337.

    Article  Google Scholar 

  28. Dargis M. Do You Hear Sleigh Bells? Nah, Just Tom Hanks and Some Train. Rev. of The Polar Express, dir. Robert Zemeckis. The New York Times. 10 Nov 2004. http://www.nytimes.com/2004/11/10/movies/10pola.html?_r=0.

  29. Tinwell A, Grimshaw M, Nabi DA, Williams A. Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Comput Hum Behav. 2011;27(2):741–9.

    Article  Google Scholar 

  30. McCloud S. Understanding comics: the invisible art. New York, NY: Kitchen Sink Press for Harper Collins; 1993.

    Google Scholar 

  31. Wallraven C, Bülthoff HH, Cunningham DW, Fischer J, Bartz D. Evaluation of real-world and computer-generated stylized facial expressions. In: TAP; ACM Transactions on Applied Perception; 2007: 4(3).

  32. Gielniak MJ, Thomaz AL. Enhancing interaction through exaggerated motion synthesis. In: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction. ACM; 2012.

  33. Takayama L, Dooley D, Ju W. Expressing thought: improving robot readability with animation principles. In: Proceedings of the 6th international conference on Human-robot interaction; ACM; 2011.

  34. Young, JE, Xin M, Sharlin E. Robot expressionism through cartooning. In: Proceedings of the ACM/IEEE international conference on Human-robot interaction; ACM; 2007.

  35. Bartneck C, Reichenbach J. Subtle emotional expressions of synthetic characters. Int J Hum Comput Stud. 2005;62(2):179–92.

    Article  Google Scholar 

  36. Tinwell A, Grimshaw M, Abdel-Nabi D. Effect of emotion and articulation of speech on the uncanny valley in virtual characters. In: Affective computing and intelligent interaction; Springer Berlin Heidelberg; 2011. p. 557–566.

  37. Kätsyri J. Human recognition of basic emotions from posed and animated dynamic facial expressions. Doctoral dissertation, Helsinki University of Technology, Espoo; 2006. http://lib.tkk.fi/Diss/2006/isbn951228538X/.

  38. Ekman P, Friesen WV, Hager JC. Facial action coding system: investigator’s guide. Salt Lake City: A Human Face; 2002.

    Google Scholar 

  39. Ho C–C, MacDorman KF. Revisiting the uncanny valley theory: developing and validating an alternative to the Godspeed indices. Comput Hum Behav. 2010;26(6):1508–18.

    Article  Google Scholar 

  40. McDonnell R, Breidt M, Bülthoff HH. Render me real?: Investigating the effect of render style on the perception of animated virtual humans. ACM Trans Graph. 2012;31(4):91.

    Article  Google Scholar 

  41. Cheetham M, Suter P, Jäncke L. The human likeness dimension of the “uncanny valley hypothesis: Behavioral and functional MRI findings. Front Hum Neurosci. 2011;5:126.

  42. Ekman P, Friesen WV. Felt, false, and miserable smiles. J Nonverbal Behav. 1982;6(4):238–52.

    Article  Google Scholar 

Download references

Acknowledgments

This study was supported by the Graduate School in User-Centered Information Technology and the aivoAALTO research project of Aalto University.

Author information

Authors and Affiliations

  1. Department of Media Technology, School of Science, Aalto University, P.O. Box 15500, 00076, Aalto, Finland

    Meeri Mäkäräinen, Jari Kätsyri & Tapio Takala

Authors
  1. Meeri Mäkäräinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jari Kätsyri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tapio Takala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meeri Mäkäräinen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mäkäräinen, M., Kätsyri, J. & Takala, T. Exaggerating Facial Expressions: A Way to Intensify Emotion or a Way to the Uncanny Valley?. Cogn Comput 6, 708–721 (2014). https://doi.org/10.1007/s12559-014-9273-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-014-9273-0

Keywords

Search

Navigation