Skip to main content
SpringerLink
Log in

Fundamentals of Agent Perception and Attention Modelling

  • Chapter
  • First Online:
Emotion-Oriented Systems

Part of the book series: Cognitive Technologies ((COGTECH))

Abstract

Perception and attention mechanisms are of great importance for entities situated within complex dynamic environments. With roles extending greatly beyond passive information services about the external environment, such mechanisms actively prioritise, augment and expedite information to ensure that the potentially relevant is made available so appropriate action can take place. Here, we describe the rationale behind endowing artificial entities, or virtual agents, with real-time perception and attention systems. We cover the fundamentals of designing and building such systems. Once equipped, the resulting agents can achieve a more substantial connection with their environment for the purposes of reacting, planning, decision making and, ultimately, behaving.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://projects.ict.usc.edu/vision/watson/

  2. 2.

    http://www.eyesweb.org

References

  • Blumberg B (1997) Go with the flow: synthetic vision for autonomous animated creatures. In: Lewis Johnson W, Hayes-Roth B. (ed) Proceedings of the first international conference on autonomous agents (Agents’97). ACM Press, New York, NY, pp 538–539

    Chapter  Google Scholar 

  • Bordeux C, Boulic R, Thalmann D (1999) An efficient and flexible perception pipeline for autonomous agents. Proceedings of Eurographics ’99, Milano, Italy, pp 23–30

    Google Scholar 

  • Bradski GR (1998) Computer vision face tracking for use in a perceptual user interface. Intel Tech J Q2:705–740

    Google Scholar 

  • Bradski GR, Kaehler A (2008) Learning OpenCV: computer vision with the openCV library. O’Reilly, Cambridge, MA

    Google Scholar 

  • Camurri A, Mazzarino B, Volpe G (2004) Analysis of expressive gesture: the eyesweb expressive gesture processing library. In: Camurri A, Volpe G. (eds) Gesture-based communication in human-computer interaction. LNAI vol 2915, Springer, Berlin, pp 460–467

    Chapter  Google Scholar 

  • Camurri A, Coletta P, Varni G, Ghisio S (2007) Developing multimodal interactive systems with EyesWeb XMI. In: Proceedings of the 2007 conference on new interfaces for musical expression, June 6–10, New York, NY, pp 305–308

    Google Scholar 

  • Castellano G Movement expressivity analysis in affective computers: from recognition to expression of emotion. Ph.D. Dissertation, Faculty of Engineering, University of Genova, Italy, February 2008

    Google Scholar 

  • Castellano G, Villalba SD, Camurri A (2007) Recognising human emotions from body movement and gesture dynamics. In: Paiva A, Prada R, Picard RW. (eds) affective computing and intelligent interaction, second international conference, ACII 2007, Lisbon, Portugal, September 12–14, 2007, Proceedings, vol 4738 of LNCS. Springer, Berlin, pp 71–82

    Chapter  Google Scholar 

  • Castellano G, Mortillaro M, Camurri A, Volpe G, Scherer K (2008) Automated analysis of body movement in emotionally expressive piano performances. Music Percept 26(2):103–119

    Article  Google Scholar 

  • Chopra-Khullar S, Badler NI (1999) Where to look? automating attending behaviors of virtual human characters. In: AGENTS ’99: Proceedings of the third annual conference on autonomous agents. ACM Press, New York, NY

    Google Scholar 

  • Colburn A, Cohen M, Drucker S (2000) The role of eye gaze in avatar mediated conversational interfaces, Tech. Report MSR-TR-2000-81, Microsoft Research

    Google Scholar 

  • Conde T, Thalmann D (2006) An integrated perception for autonomous virtual agents: active and predictive perception. Comput Animation amp Virtual Worlds 17(3–4):457–68

    Article  Google Scholar 

  • Courty N, Marchand E (2003) Visual perception based on salient features. In: IEEE international conference on intelligent robots and systems, IROS’03, vol 2, Las Vegas, Nevada, October 2003, pp 1024–1029

    Google Scholar 

  • Delgado-Mata C, Aylett R (2001) Communicating emotion in virtual environments through artificial scents. In: IVA ’01: Proceedings of the third international workshop on intelligent virtual agents. Springer, London

    Google Scholar 

  • Eriksen CW, Yeh YY (1985) Allocation of attention in the visual field. J Exp Psychol Hum Percept Perform 11(5):583–597

    Article  Google Scholar 

  • Garau M, Slater M, Bee S, Sasse MA (2001) The impact of eye gaze on communication using humanoid avatars. In: CHI ’01: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press, New York, NY, pp 309–316

    Google Scholar 

  • Gillies M (2001) Practical behavioural animation based on vision and attention. PhD dissertation, University of Cambridge Computer Laboratory

    Google Scholar 

  • Gu E, Wang J, Badler NI (2005) Generating sequence of eye fixations using decision-theoretic attention model. IEEE computer society conference on computer vision and pattern recognition (CVPRW’05) – Workshops, p 92

    Google Scholar 

  • Herrero P, de Antonio A (2003) Introducing human-like hearing perception in intelligent virtual agents. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems (AAMAS), July 14–18, Melbourne, Australia

    Google Scholar 

  • Hill R (2000) Perceptual attention in virtual humans: towards realistic and believable gaze behaviours, AAAI Technical Report FS-00-03, pp 46–52

    Google Scholar 

  • Isla D, Burke R, Downie M, Blumberg B (2001) A layered brain architecture for synthetic creatures. In: Nebel B. (ed) Proceedings of the seventeenth international joint conference on artificial intelligence, IJCAI 2001, Seattle, Washington, USA, August 4–10, 2001, Morgan Kaufmann, 2001, pp 1051–1058

    Google Scholar 

  • Itti L, Koch C, Niebur E (1998 November) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Google Scholar 

  • Itti L, Dhavale N, Pighin F (2003) Realistic avatar eye and head animation using a neurobiological model of visual attention. In: Proceedings of the SPIE 48th annual international symposium on optical science and technology, San Diego, USA, August 3–8, pp 64–78

    Google Scholar 

  • Kahneman D, Henrik A (1981) Perceptual organization, chapter perceptual organization and attention. Erlbaum, Hillsdale, NJ

    Google Scholar 

  • Kim Y, Van Velsen M, Hill R (2005) Modeling dynamic perceptual attention in complex virtual environments. In: Panayiotopoulos T, Gratch J, Aylett R, Ballin D, Olivier P, Rist T. (eds) Intelligent virtual agents, 5th international working conference, IVA 2005, Kos, Greece, September 12–14, 2005, Proceedings. Lecture notes in computer science, vol 3661. Springer, London, pp 266–277

    Google Scholar 

  • Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiol 4:219–227

    Google Scholar 

  • Kranstedt A, Lücking A, Pfeiffer T, Rieser H, Wachsmuth I (2006) Deixis: how to determine demonstrated objects using a pointing cone. In: Gibet S, Courty N, Kamp JF (eds) Gesture in human-computer interaction and simulation, vol 3881. Springer, New York, NY, pp 300–301

    Chapter  Google Scholar 

  • Kuffner J, Latombe J-C (1999 May) Fast synthetic vision, memory, and learning models for virtual humans. In: Proceedings of computer animation ’99. IEEE Computer Society, Washington, DC

    Google Scholar 

  • Laptev I, Lindeberg T (2003) Space-time interest points. In: Proceedings of ICCV03, Nice, France, October 13–16, pp 432–443

    Google Scholar 

  • Lee SP, Badler JB, Badler NI (2002) Eyes alive. In: Proceedings of the 29th annual conference on computer graphics and interactive techniques (SIGGRAPH ’02). ACM Press, New York, NY, pp 637–644

    Google Scholar 

  • Leonard T (2003) Building an AI sensory system – examining the design of thief - the dark project. In: Proceedings of game developers conference 2003, CMP Game Media Group San Francisco, CA

    Google Scholar 

  • Lozano M, Lucia R, Barber F, Grimaldo F, Soares AL, Fornes A (2003) An efficient synthetic vision system for 3d multi-character systems. In: Rist T, Aylett R, Ballin D, Rickel J (eds) Intelligent agents, 4th international workshop, IVA 2003, Kloster Irsee, Germany, September 15–17, 2003, Proceedings, vol 2792 of Lecture notes in computer science. Springer, New York, NY, pp 356–357

    Google Scholar 

  • Morency L-P, Darrell T (2004) From conversational tooltips to grounded discourse: head pose tracking in interactive dialog systems. In: Proceedings of the 6th international conference on multimodal interfaces, State College, PA, October 2004, pp 32–37

    Google Scholar 

  • Neisser U (1967) Cognitive psychology. Appleton-Century-Crofts New York, NY

    Google Scholar 

  • Noser H, Renault O, Thalmann D, Thalmann NM (1995) Navigation for digital actors based on synthetic vision, memory and learning. Comput Graph 19(1):7–19

    Article  Google Scholar 

  • Oikonomopoulos A, Patras I, Pantic M (2006 June) Spatiotemporal salient points for visual recognition of human actions. IEEE Trans Syst Man, Cybern B Cybern 36(3):710–719

    Article  Google Scholar 

  • Ouerhani N, von Wartburg R, Hugli H (2004) Empirical validation of the saliency-based model of visual attention. Electron Lett Comput Vis Image Anal 3(1):13–24

    Google Scholar 

  • Pantic M, Bartlett MS (2007) Machine analysis of facial expressions. In: Delac K, Grgic M (eds) Face recognition. I-Tech Education and Publishing, Vienna, Austria, pp 377–416

    Google Scholar 

  • Patras I, Pantic M (2004) Particle filtering with factorized likelihoods for tracking facial features. In: Proceedings of the IEEE international conference on face and gesture recognition. Seoul, Korea, May 17–19, pp 97–102

    Google Scholar 

  • Peters C (2006) Evaluating perception of interaction initiation in virtual environments using humanoid agents. In: Proceedings of the 17th European conference on artificial intelligence, Riva Del Garda, Italy, August 2006, pp 46–50

    Google Scholar 

  • Peters C, O’Sullivan C (2002) Synthetic vision and memory for autonomous virtual humans. Comput Graph Forum 21(4):743–743

    Article  Google Scholar 

  • Peters C, O’Sullivan C (2003) Bottom-up visual attention for virtual human animation. In: CASA ’03: proceedings of the 16th international conference on computer animation and social agents (CASA 2003), IEEE Computer Society, Washington, DC, p 111

    Google Scholar 

  • Peters RJ, Itti L (2008) Applying computational tools to predict gaze direction in interactive visual environments. ACM Trans Appl Percept 5(2):8

    Article  Google Scholar 

  • Picot A, Bailly G, Elisei F, Raidt S (2007) Scrutinizing natural scenes: controlling the gaze of an embodied conversational agent. In: Pelachaud C, Martin J-C, André E, Chollet G, Karpouzis K, Pelé D (eds) Intelligent virtual agents 2007, Proceedings, LNCS, vol 4722. Springer, Berlin, pp 272–282

    Chapter  Google Scholar 

  • Poggi I, Pelachaud C, de Rosis F (2000) Eye communication in a conversational 3d synthetic agent. AI Commun 13(3):169–182

    Google Scholar 

  • Posner MI (1980) Orienting of attention. Quart J Exp Psychol 32:3–25

    Article  Google Scholar 

  • Privitera H, Stark LW (2000) Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Trans Pattern Anal Mach Intell (PAMI) 22(9):970–982

    Article  Google Scholar 

  • Rabiner LR (1990) A tutorial on hidden Markov models and selected applications in speech recognition. In: Waibel A, Lee K-F (eds) Readings in speech recognition.  Morgan Kaufmann Publishers  Inc., San Francisco, CA, pp 267–296

    Google Scholar 

  • Raidt S, Bailly G, Elisei F (2005) Basic components of a face-to-face interaction with a conversational agent: mutual attention and deixis. Proceedings of the 2005 joint conference on smart objects and ambient intelligence: innovative context-aware services: usages and technologies, Grenoble, France, October 12–14, pp 247–252

    Google Scholar 

  • Rapantzikos K, Avrithis Y, Kollias S (2005) Handling uncertainty in video analysis with spatiotemporal visual attention. In: Proceedings of IEEE international conference on fuzzy systems, Reno, Nevada, May 2005

    Google Scholar 

  • Rapantzikos K, Tsapatsoulis N, Avrithis Y, Kollias S (June 2007) A bottom-up spatiotemporal visual attention model for video analysis. IET Image process 1(2):237–248

    Article  Google Scholar 

  • Rehm M, André E (2005) Where do they look?: gaze behaviors of multiple users interacting with an embodied conversational agent, vol 3661, pp 241–252

    Google Scholar 

  • Reynolds CW (2000) Interaction with groups of autonomous characters. In: Proceedings of game developers conference 2000. CMP Game Media Group, San Francisco, CA, pp 449–460

    Google Scholar 

  • Scherer K, Schorr A, Johnstone T (February 2001) Appraisal processes in emotion: theory, methods, research (series in affective science). Oxford University Press, Bethesda, MD

    Google Scholar 

  • Terzopoulos D, Rabie TF (1997) Animat vision: active vision in artificial animals. Videre J Comput Vis Res 1(1):2–19

    Google Scholar 

  • Thórisson KR (1997) Gandalf: an embodied humanoid capable of real-time multimodal dialogue with people. In: AGENTS ’97: Proceedings of the first international conference on autonomous agents. ACM Press, New York, NY, pp 536–537

    Google Scholar 

  • Treisman AM, Gelade G (1980) A feature integration theory of attention. Cogn Psychol 12:97–136

    Article  Google Scholar 

  • Tu X, Terzopoulos D (1994) Artificial fishes: physics, locomotion, perception, behavior. In: SIGGRAPH ’94: Proceedings of the 21st annual conference on computer graphics and interactive techniques. ACM Press, New York, NY, pp 43–50

    Google Scholar 

  • van Es I, Heylen D, van Dijk B, Nijholt A (2002) Making agents gaze naturally – does it work? Proceedings of the working conference on advanced visual interfaces, AVI ’02, Trento, Italy, pp 357–358, ACM, New York, NY

    Google Scholar 

  • Vertegaal R, Ding Y (2002) Explaining effects of eye gaze on mediated group conversations: amount or synchronization? In: Churchill EF, McCarthy J, Neuwirth C, Rodden T (eds), CSCW ’02: proceedings of the 2002 ACM conference on computer supported cooperative work. ACM Press, New York, NY, pp 41–48

    Google Scholar 

  • Viola P, Jones MJ (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol 1. Kauai, Hawaii, pp 511–518, December 8–14, IEEE Computer Society, Los Alamitos, CA

    Google Scholar 

  • Vosinakis S, Panayiotopoulos T (2003) Programmable agent perception in intelligent virtual environments. In: Rist T, Aylett R, Ballin D, Rickel J (eds) Intelligent agents, 4th international workshop, IVA 2003, Kloster Irsee, Germany, September 15–17, 2003, proceedings, vol 2792 of Lecture notes in computer science. Springer, London, UK

    Google Scholar 

  • Yarbus A (1967) Eye movements and vision, chapter Eye movements during perception of complex objects. Plenum Press, New York, NY

    Google Scholar 

  • Zeng Z, Pantic M, Roisman G, Huang T (2009 January) A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Coventry University, Conventry, UK

    Christopher Peters

  2. Queen Mary University of London, London, UK

    Ginvera Castellano

  3. University of Augsburg, Augsburg, Germany

    Matthias Rehm & Elisabeth André

  4. National Technical University of Athens, Athens, Greece

    Amaryllis Raouzaiou & Kostas Rapantzikos

  5. Image, Video and Multimedia Systems Lab, Institute of Communications and Computer Systems, National Technical University of Athens, Athens, Greece

    Kostas Karpouzis

  6. University of Genova, Genoa, Italy

    Gaultiero Volpe & Antonio Camurri

  7. University of Bath, Bath, UK

    Asimina Vasalou

Authors
  1. Christopher Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ginvera Castellano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthias Rehm
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elisabeth André
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amaryllis Raouzaiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kostas Rapantzikos
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kostas Karpouzis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Gaultiero Volpe
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Antonio Camurri
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Asimina Vasalou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher Peters .

Editor information

Editors and Affiliations

  1. , Department of Psychology, Queen's University Belfast, Belfast, BT7 1NN, United Kingdom

    Roddy Cowie

  2. TELECOM ParisTech, CNRS - LTCI, rue Dareau 37, Paris, 75014, France

    Catherine Pelachaud

  3. for Artificial Intelligence (OFAI), Austrian Research Institute, Freyung 6/6, Vienna, 1010, Austria

    Paolo Petta

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Peters, C. et al. (2011). Fundamentals of Agent Perception and Attention Modelling. In: Cowie, R., Pelachaud, C., Petta, P. (eds) Emotion-Oriented Systems. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15184-2_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-15184-2_16

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-15183-5

  • Online ISBN: 978-3-642-15184-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation