Skip to main content
SpringerLink
Log in

Large-Scale Observational Evidence of Cross-Cultural Differences in Facial Behavior

  • Original Paper
  • Published:
Journal of Nonverbal Behavior Aims and scope Submit manuscript

Abstract

Self-report studies have found evidence that cultures differ in the display rules they have for facial expressions (i.e., for what is appropriate for different people at different times). However, observational studies of actual patterns of facial behavior have been rare and typically limited to the analysis of dozens of participants from two or three regions. We present the first large-scale evidence of cultural differences in observed facial behavior, including 740,984 participants from 12 countries around the world. We used an Internet-based framework to collect video data of participants in two different settings: in their homes and in market research facilities. Using computer vision algorithms designed for this dataset, we measured smiling and brow furrowing expressions as participants watched television ads. Our results reveal novel findings and provide empirical evidence to support theories about cultural and gender differences in display rules. Participants from more individualist cultures displayed more brow furrowing overall, whereas smiling depended on both culture and setting. Specifically, participants from more individualist countries were more expressive in the facility setting, while participants from more collectivist countries were more expressive in the home setting. Female participants displayed more smiling and less brow furrowing than male participants overall, with the latter difference being more pronounced in more individualist countries. This is the first study to leverage advances in computer science to enable large-scale observational research that would not have been possible using traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The base rate is the proportion of time that a participant displays a facial action.

References

  • Ansfield, M. E. (2007). Smiling when distressed: When a smile is a frown turned upside down. Personality and Social Psychology Bulletin, 33(6), 763–775.

    Article  PubMed  Google Scholar 

  • Argyle, M., Henderson, M., Bond, M., Iizuka, Y., & Contarello, A. (1986). Cross-cultural variations in relationship rules. International Journal of Psychology, 21(1–4), 287–315.

    Article  Google Scholar 

  • Brennan, R. L., & Prediger, D. J. (1981). Coefficient Kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41(3), 687–699.

    Article  Google Scholar 

  • Brewer, M. B., & Kramer, R. M. (1985). The psychology of intergroup attitudes and behavior. Annual Review of Psychology, 36(1), 219–243.

    Article  Google Scholar 

  • Brody, L. R., & Hall, J. A. (2008). Gender and emotion in context. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of emotions (3rd ed., pp. 395–408). New York, NY: The Guilford Press.

    Google Scholar 

  • Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., & Robinson, E. S. J. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376.

    Article  PubMed  Google Scholar 

  • Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. Journal of Personality and Social Psychology, 50(2), 260.

    Article  PubMed  Google Scholar 

  • Cacioppo, J. T., & Tassinary, L. G. (1990). Inferring psychological significance from physiological signals. The American Psychologist, 45(1), 16–28.

    Article  PubMed  Google Scholar 

  • Camras, L. A., Bakeman, R., Chen, Y., Norris, K., & Cain, T. R. (2006). Culture, ethnicity, and children’s facial expressions: A study of European American, Mainland Chinese, Chinese American, and adopted Chinese girls. Emotion, 6(1), 103–114.

    Article  PubMed  Google Scholar 

  • Camras, L. A., Oster, H., Campos, J., Campos, R., Ujiie, T., Miyake, K., et al. (1998). Production of emotional facial expressions in European American, Japanese, and Chinese infants. Developmental Psychology, 34(4), 616–628.

    Article  PubMed  Google Scholar 

  • Casella, G., & George, E. I. (1992). Explaining the gibbs sampler. The American Statistician, 46(3), 167–174.

    Google Scholar 

  • Cohn, J. F., & Ekman, P. (2005). Measuring facial action by manual coding, facial EMG, and automatic facial image analysis. In J. A. Harrigan, R. Rosenthal, & K. R. Scherer (Eds.), The new handbook of nonverbal behavior research (pp. 9–64). New York, NY: Oxford University Press.

    Google Scholar 

  • Darwin, C. (1872). The expression of emotions in man and animals (3rd ed.). New York, NY: Oxford University.

    Book  Google Scholar 

  • Ekman, P. (1977). Biological and cultural contributions to body and facial movement. In J. Blacking (Ed.), The anthropology of the body. London: Academic Press.

    Google Scholar 

  • Ekman, P. (1979). About brows: Emotional and conversational signals. In M. V. Cranach, K. Foppa, W. Lepenies, & D. Ploog (Eds.), Human ethology (pp. 169–202). Cambridge: Cambridge University Press.

    Google Scholar 

  • Ekman, P. (1982). Methods for measuring facial action. In K. R. Scherer & P. Ekman (Eds.), Handbook of methods in nonverbal behavior research (pp. 45–90). Cambridge: Cambridge University Press.

    Google Scholar 

  • Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne smile: Emotional expression and brain physiology: II. Journal of Personality and Social Psychology, 58(2), 342–353.

    Article  PubMed  Google Scholar 

  • Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17, 124–129.

    Article  PubMed  Google Scholar 

  • Ekman, P., Friesen, W. V., & Ellsworth, P. (1972). Emotion in the human face: Guidelines for research and an integration of findings. New York, NY: Pergamon Press.

    Google Scholar 

  • Ekman, P., Friesen, W. V., & Hager, J. (2002). Facial action coding system: A technique for the measurement of facial movement. Salt Lake City, UT: Research Nexus.

    Google Scholar 

  • Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(4), 712–717.

    Article  PubMed  Google Scholar 

  • Ekman, P., & Rosenberg, E. L. (2005). What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (FACS) (2nd ed.). New York, NY: Oxford University Press.

    Book  Google Scholar 

  • Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science, 164(3875), 86–88.

    Article  PubMed  Google Scholar 

  • Evans, E. C. (1969). Physiognomies in the ancient world. Transactions of the American Philosophical Society, 59, 1–101.

    Article  Google Scholar 

  • Fischer, A. H., & Manstead, A. S. R. (2000). The relation between gender and emotions in different cultures. In A. H. Fischer (Ed.), Gender and emotion: Social psychological perspectives (pp. 71–94). New York, NY: Cambridge University Press.

    Chapter  Google Scholar 

  • Fridlund, A. J. (1991). Sociality of solitary smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology, 60(2), 229–240.

    Article  Google Scholar 

  • Friesen, W. V. (1973). Cultural differences in facial expressions in a social situation: An experimental test on the concept of display rules (Doctoral dissertation). University of California San Francisco.

  • Frith, H., & Kitzinger, C. (1998). “Emotion work” as a participant re-source: A feminist analysis of young women’s talk-in-interaction. Sociology, 32, 299–320.

    Article  Google Scholar 

  • Girard, J. M., & Cohn, J. F. (2016). A primer on observational measurement. Assessment, 23(4), 404–413.

    Article  PubMed  Google Scholar 

  • Girard, J. M., Cohn, J. F., Jeni, L. A., Sayette, M. A., & De la Torre, F. (2015). Spontaneous facial expression in unscripted social interactions can be measured automatically. Behavior Research Methods, 47(4), 1136–1147.

    Article  PubMed  PubMed Central  Google Scholar 

  • Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Hammal, Z., & Rosenwald, D. P. (2014). Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses. Image and Vision Computing, 32(10), 641–647.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gosling, S. D., John, O. E., Craik, K. H., & Robins, R. W. (1998). Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. Journal of Personality and Social Psychology, 74(5), 1337–1349.

    Article  PubMed  Google Scholar 

  • Hall, J. A., & Halberstadt, A. G. (1986). Smiling and gazing. In J. S. Hyde & M. C. Inn (Eds.), The psychology of gender: Advances through meta-analysis (pp. 136–185). Baltimore, MD: Johns Hopkins University Press.

    Google Scholar 

  • Hecht, M. A., & LaFrance, M. (1998). License or obligation to smile: The effect of power and sex on amount and type of smiling. Personality and Social Psychology Bulletin, 24(12), 1332–1342.

    Article  Google Scholar 

  • Heck, R. H., & Thomas, S. L. (2015). An introduction to multilevel modeling techniques: MLM and SEM approaches using Mplus (3rd ed.). New York, NY: Routledge.

    Google Scholar 

  • Hess, U., Adams, R. B., Jr., & Kleck, R. E. (2005). Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition and Emotion, 19(4), 515–536.

    Article  Google Scholar 

  • Hofstede, G. H. (1980). Culture’s consequences: International differences in work-related values. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  • Hofstede, G. H. (2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations (2nd ed.). Thousand Oaks: Sage Publications.

    Google Scholar 

  • Hofstede, G. H., & McCrae, R. R. (2004). Personality and culture revisited: Linking traits and dimensions of culture. Cross-Cultural Research, 38(1), 52–88.

    Article  Google Scholar 

  • Kagan, J., Arcus, D., Snidman, N., Feng, W. Y., Hendler, J., & Greene, S. (1994). Reactivity in infants: A cross-national comparison. Developmental Psychology, 30(3), 342–345.

    Article  Google Scholar 

  • Kisilevsky, B. S., Hains, S. M. J., Lee, K., Muir, D. W., Xu, F., Fu, G., et al. (1998). The still-face effect in Chinese and Canadian 3- to 6-month-old infants. Developmental Psychology, 34(4), 629–639.

    Article  PubMed  Google Scholar 

  • Klineberg, O. (1940). Social psychology. New York, NY: Holt.

    Book  Google Scholar 

  • LaFrance, M., Hecht, M. A., & Paluck, E. L. (2003). The contingent smile: A meta-analysis of sex differences in smiling. Psychological Bulletin, 129(2), 305–334.

    Article  PubMed  Google Scholar 

  • Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98(2), 224–253.

    Article  Google Scholar 

  • Matsumoto, D. (1990). Cultural similarities and differences in display rules. Motivation and Emotion, 14(3), 195–214.

    Article  Google Scholar 

  • Matsumoto, D. (2006). Culture and nonverbal behavior. In Handbook of nonverbal communication (pp. 219–236).

  • Matsumoto, D., & Kupperbusch, C. (2001). Idiocentric and allocentric differences in emotional expression, experience, and the coherence between expression and experience. Asian Journal of Social Psychology, 4(2), 113–131.

    Article  Google Scholar 

  • Matsumoto, D., & Willingham, B. (2006). The thrill of victory and the agony of defeat: Spontaneous expressions of medal winners of the 2004 Athens Olympic games. Journal of Personality and Social Psychology, 91(3), 568–581.

    Article  PubMed  Google Scholar 

  • Matsumoto, D., Willingham, B., & Olide, A. (2009). Sequential dynamics and culturally-moderated facial expressions of emotion. Psychological Science, 20(10), 1269–1274.

    Article  PubMed  Google Scholar 

  • Matsumoto, D., Yoo, S. H., & Fontaine, J. (2008). Mapping expressive differences around the world: The relationship between emotional display rules and individualism versus collectivism. Journal of Cross-Cultural Psychology, 39(1), 55–74.

    Article  Google Scholar 

  • McCrae, R. R., & Costa, P. T. (1999). A five-factor theory of personality. In L. A. Pervin & O. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 139–153). New York, NY: Guilford Press.

    Google Scholar 

  • McDuff, D. (2014). Crowdsourcing affective responses for predicting media effectiveness (Doctoral dissertation). Massachusetts Institute of Technology.

  • McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.

    Article  Google Scholar 

  • Messick, D. M., & Mackie, D. M. (1989). Intergroup relations. Annual Review of Psychology, 40, 45–81.

    Article  PubMed  Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.

    Google Scholar 

  • Oster, H. (1978). Facial expression and affect development. In M. Lewis & L. A. Rosenblum (Eds.), The development of affect. New York, NY: Plenum Press.

    Google Scholar 

  • Oster, H., Daily, L., & Goldenthal, P. (1989). Processing facial affect. In A. W. Young & H. D. Ellis (Eds.), Handbook of research on face processing (pp. 101–161). Amsterdam: Elsevier.

    Google Scholar 

  • Oyserman, D., Coon, H. M., & Kemmelmeier, M. (2002). Rethinking individualism and collectivism: Evaluation of theoretical assumptions and meta-analyses. Psychological Bulletin, 128(1), 3.

    Article  PubMed  Google Scholar 

  • Russell, J. A. (1994). Is there universal recognition of emotion from facial expressions? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102.

    PubMed  Google Scholar 

  • Rychlowska, M., Miyamoto, Y., Matsumoto, D., Hess, U., Gilboa-Schechtman, E., Kamble, S., et al. (2015). Heterogeneity of long-history migration explains cultural differences in reports of emotional expressivity and the functions of smiles. Proceedings of the National Academy of Sciences, 112(19), E2429–E2436.

    Article  Google Scholar 

  • Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.

    Article  PubMed  Google Scholar 

  • Scherer, K. R., Wallbott, H. F., & Summerfield, A. B. (1986). Experiencing emotion: A cross-cultural study. New York, NY: Cambridge University Press.

    Google Scholar 

  • Schwartz, G. E., Ahern, G. L., & Brown, S. L. (1979). Lateralized facial muscle response to positive and negative emotional stimuli. Psychophysiology, 16(6), 561–571.

    Article  PubMed  Google Scholar 

  • Senechal, T., McDuff, D., & Kaliouby, R. (2015). Facial action unit detection using active learning and an efficient non-linear kernel approximation. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 10–18).

  • Singelis, T. M. (1994). The measurement of independent and interdependent self-construals. Personality and Social Psychology Bulletin, 20(5), 580–591.

    Article  Google Scholar 

  • Suh, E., Diener, E., Oishi, S., & Triandis, H. C. (1998). The shifting basis of life satisfaction judgments across cultures: Emotions versus norms. Journal of Personality and Social Psychology, 74(2), 482–493.

    Article  Google Scholar 

  • Tajfel, H. (1982). Social psychology of intergroup relations. Annual Review of Psychology, 33(1), 1–39.

    Article  Google Scholar 

  • Waxer, P. H. (1985). Video ethology: Television as a data base for cross-cultural studies in nonverbal displays. Journal of Nonverbal Behavior, 9(2), 111–120.

    Article  Google Scholar 

Download references

Author information

Author notes

    Authors and Affiliations

    1. Affectiva, Inc., 463 Waverley Oaks Road, Waltham, MA, 02452, USA

      Daniel McDuff & Rana el Kaliouby

    2. Department of Psychology, University of Pittsburgh, Pittsburgh, PA, 15260, USA

      Jeffrey M. Girard

    Authors
    1. Daniel McDuff
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Jeffrey M. Girard
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Rana el Kaliouby
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding authors

    Correspondence to Daniel McDuff or Jeffrey M. Girard.

    Additional information

    Daniel McDuff and Jeffrey M. Girard have contributed equally to this study.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    McDuff, D., Girard, J.M. & Kaliouby, R.e. Large-Scale Observational Evidence of Cross-Cultural Differences in Facial Behavior. J Nonverbal Behav 41, 1–19 (2017). https://doi.org/10.1007/s10919-016-0244-x

    Download citation

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s10919-016-0244-x

    Keywords

    Search

    Navigation