Abstract
This paper investigates to which extent participants in spontaneously occurring interactions can be recognised automatically from the shape description of their bodily behaviours. For this purpose, we apply classification algorithms to an annotated corpus of Danish dyadic and triadic conversations. The bodily behaviours which we consider are head movement, facial expressions and hand gestures. Although the data used are of limited size, the results of classification are promising especially for hand gestures indicating big variance in people’s bodily behaviours even if the involved participants are a homogeneous group in terms of gender, age and social background. The obtained results are not only interesting from a theoretic point of view, but they can also be relevant for video indexing and searching, computer games and other applications which involve multimodal interaction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Idris, F., Panchanathan, S.: Review of Image and Video Indexing Techniques. J. Vis. Commun. Image R 8(2), 146–166 (1997)
Beigi, S.: Fundamentals of Speaker Recognition. Springer (2011)
Erzin, E., Yemez, Y., Tekalp, A.M.: Multimodal Speaker Identification Using an Adaptive Classifier Cascade Based on Modality Reliability. IEEE Trans. Multimedia 7(5), 840–852 (2005)
Wu, Z., Cai, L., Meng, H.: Multi-level fusion of audio and visual features for speaker identification. In: International Conference on Advances in Biometrics, pp. 493–499 (2006)
Wee-Chung, A., Wang, L., Wang, S. (ed.): Speech Recognition: Lip Segmentation and Mapping. IGI Global (2009)
Navarretta, C.: Anaphora and gestures in multimodal communication. In: Hendrickx, Branco, Devi, L., Mitkov (eds.) Proceedings of the 8th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 2011), Faro, Portugal, Edicoes Colibri, 171–181 (2011)
Paggio, P., Navarretta, C.: Feedback in Head Gestures and Speech. In: Kipp, M., et al. (eds.) LREC 2010 Workshop Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality, pp. 1–4 (2010)
Navarretta, C., Paggio, P.: Classification of Feedback Expressions in Multimodal Data. In: Proceedings of ACL 2010, Uppsala, Sweden, pp. 318–324 (2010)
Paggio, P., Navarretta, C.: C. Learning to classify the feedback function of head movements in a Danish Corpus of first encounters. In: Proceedings of ICMI 2011 Workshop Multimodal Corpora for Machine Learning: Taking Stock and Road mapping the Future, Alicante, Spain, pp. 49–54 (November 2011)
Navarretta, C.: Annotating Non-verbal Behaviours in Informal Interactions. In: Esposito, A., Vinciarelli, A., Vicsi, K., Pelachaud, C., Nijholt, A. (eds.) Communication and Enactment 2010. LNCS, vol. 6800, pp. 309–315. Springer, Heidelberg (2011)
MacWhinney, B., Wagner, J.: Transcribing, searching and data sharing: The CLAN software and the TalkBank data repository. Gespraechsforschung 11, 154–173 (2010)
Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., Paggio, P.: The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. In: Martin, J.-C., et al. (eds.) Multimodal Corpora for Modelling Human Multimodal Behaviour, Special issue of the International JLRE, pp. 273–287. Springer (2007)
Peirce, C.S.: Collected Papers of Charles Sanders Peirce, 1931-1958, 8 vols, Hartshorne, C., Weiss, P., Burks, A. (eds.). Harvard University Press, Cambridge (1931)
Duncan, S.: McNeill Lab Coding Methods. Technical Report (2004)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)
Jokinen, K., Navarretta, C., Paggio, P.: Distinguishing the Communicative Functions of Gestures. In: Popescu-Belis, A., Stiefelhagen, R. (eds.) MLMI 2008. LNCS, vol. 5237, pp. 38–49. Springer, Heidelberg (2008)
Navarretta, C., Ahlseen, E., Allwood, J., Jokinen, K., Paggio, P.: Creating Comparable Multimodal Corpora for Nordic Languages. In: Proceedings of Nodalida 2011, pp. 153–160 (2011)
Lu, J., Allwood, J., Ahlsén, E.: A Study on Cultural Variations of Smile Based on Empirical Recordings of Chinese and Swedish First Encounters. In: Proceedings of ICMI 2011 Workshop Multimodal Corpora for Machine Learning: Taking Stock and Road mapping the Future, Alicante, Spain, pp. 37–42 (November 2011)
Jokinen, K.: Turn taking, Utterance Density, and Gaze Patterns as Cues to Conversational Activity. In: Proceedings of ICMI 2011 Workshop Multimodal Corpora for Machine Learning: Taking Stock and Road mapping the Future, Alicante, Spain, pp. 31–36 (November 2011)
Kipp, M.: Gesture Generation by Imitation - From Human Behavior to Computer Character Animation. Ph.D. thesis, Saarland University, Saarbruecken, Germany, Boca Raton, Florida, dissertation.com (2004)
Witten, J.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Platt, J.C.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Schoelkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 41–65. MIT Press (1998)
Gallaher, P.E.: Individual differences in nonverbal behavior: Dimensions of style. J. Pers. Soc. Psychol. 63(1), 133–145 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Navarretta, C. (2012). Individuality in Communicative Bodily Behaviours. In: Esposito, A., Esposito, A.M., Vinciarelli, A., Hoffmann, R., Müller, V.C. (eds) Cognitive Behavioural Systems. Lecture Notes in Computer Science, vol 7403. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34584-5_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-34584-5_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34583-8
Online ISBN: 978-3-642-34584-5
eBook Packages: Computer ScienceComputer Science (R0)