Skip to main content
SpringerLink
Log in

Emotional Intelligence in Robotics: A Scoping Review

  • Conference paper
  • First Online:
New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence (DiTTEt 2021)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1410))

Abstract

Research suggests that emotionally responsive machines that can simulate empathy increase de acceptance of users towards them, as the feeling of affinity towards the machine reduces negative perceptual feedback. In order to endow a robot with emotional intelligence, it must be equipped with sensors capable of capturing users’ emotions (sense), appraisal captured emotions to regulate its internal state (compute), and finally perform tasks where actions are regulated by the computed “emotional” state (act). However, despite the impressive progress made in recent years in terms of artificial intelligence, speech recognition and synthesis, computer vision and many other disciplines directly and indirectly related to artificial emotional recognition and behavior, we are still far from being able to endow robots with the empathic capabilities of a human being. This article aims to give an overview of the implications of introducing emotional intelligence in robotic constructions by discussing recent advances in emotional intelligence in robotics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aranha, R.V., Corrêa, C.G., Nunes, F.L.S.: Adapting software with affective computing: a systematic review. IEEE Trans. Affect. Comput. 1 (2019). https://doi.org/10.1109/TAFFC.2019.2902379

  2. Yadegaridehkordi, E., Noor, N.F.B.M., Ayub, M.N.B., Affal, H.B., Hussin, N.B.: Affective computing in education: a systematic review and future research. Comput. Educ. 142, 103649 (2019). https://doi.org/10.1016/j.compedu.2019.103649

  3. Dudek, M., Baisch, S., Knopf, M., Kolling, T.: This isn’t me!: the role of age-related self- and user images for robot acceptance by elders. Int. J. Soc. Robot. (2020). https://doi.org/10.1007/s12369-020-00678-1

    Article  Google Scholar 

  4. Mele, C., et al.: Understanding robot acceptance/rejection: the SAR model. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 470–475 (2020). https://doi.org/10.1109/RO-MAN47096.2020.9223577

  5. Meissner, A., Trübswetter, A., Conti-Kufner, A.S., Schmidtler, J.: Friend or foe? Understanding assembly workers 2019; acceptance of human-robot collaboration. ACM Trans. Hum.-Robot Interacti. 10(1), 3:1–3:30. https://doi.org/10.1145/3399433

  6. Savery, R., Weinberg, G.: A survey of robotics and emotion: classifications and models of emotional interaction. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 986–993 (2020). https://doi.org/10.1109/RO-MAN47096.2020.9223536

  7. Salovey, P., Mayer, J.D.: Emotional intelligence. Imagin. Cogn. Pers. 9(3), 185–211 (1990). https://doi.org/10.2190/DUGG-P24E-52WK-6CDG

    Article  Google Scholar 

  8. What is a robot? - ROBOTS: your guide to the world of robotics. (n.d.). https://robots.ieee.org/learn/what-is-a-robot/. Accessed: 28 Apr 2021

  9. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980). https://doi.org/10.1037/h0077714

    Article  Google Scholar 

  10. Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018). https://doi.org/10.3390/s18072074

    Article  Google Scholar 

  11. van der Kruk, E., Reijne, M.M.: Accuracy of human motion capture systems for sport applications; state-of-the-art review. Eur. J. Sport Sci. 18(6), 806–819 (2018). https://doi.org/10.1080/17461391.2018.1463397

    Article  Google Scholar 

  12. Rohlfing, M.L., Buckley, D.P., Piraquive, J., Stepp, C.E., Tracy, L.F.: Hey Siri: How effective are common voice recognition systems at recognizing dysphonic voices? The Laryngoscope (2021). https://doi.org/10.1002/lary.29082

  13. Samadiani, N., et al.: A review on automatic facial expression recognition systems assisted by multimodal sensor data. Sensors 19(8), 1863 (2019). https://doi.org/10.3390/s19081863

    Article  Google Scholar 

  14. Khalil, R.A., Jones, E., Babar, M.I., Jan, T., Zafar, M.H., Alhussain, T.: Speech emotion recognition using deep learning techniques: a review. IEEE Access 7, 117327–117345 (2019). https://doi.org/10.1109/ACCESS.2019.2936124

    Article  Google Scholar 

  15. Abdullah, S.M.S.A., Ameen, S.Y.A., Sadeeq, M.A.M., Zeebaree, S.: Multimodal emotion recognition using deep learning. J. Appl. Sci. Technol. Trends 2(2), 52–58 (2021). https://doi.org/10.38094/jastt20291

  16. Tzirakis, P., Chen, J., Zafeiriou, S., Schuller, B.: End-to-end multimodal affect recognition in real-world environments. Inf. Fusion 68, 46–53 (2021). https://doi.org/10.1016/j.inffus.2020.10.011

    Article  Google Scholar 

  17. Li, S., et al.: Bi-modality fusion for emotion recognition in the wild. In: 2019 International Conference on Multimodal Interaction, pp. 589–594 (2019). https://doi.org/10.1145/3340555.3355719

  18. Moerland, T.M., Broekens, J., Jonker, C.M.: Emotion in reinforcement learning agents and robots: a survey. Mach. Learn. 107(2), 443–480 (2017). https://doi.org/10.1007/s10994-017-5666-0

    Article  MathSciNet  Google Scholar 

  19. Zhou, Q.: Multi-layer affective computing model based on emotional psychology. Electron. Commer. Res. 18(1), 109–124 (2017). https://doi.org/10.1007/s10660-017-9265-8

    Article  Google Scholar 

  20. Calvo, R.: The Oxford Handbook of Affective Computing. Oxford University Press (2015). https://doi.org/10.1093/oxfordhb/9780199942237.001.0001

  21. Broekens, J., Bosse, T., Marsella, S.C.: Challenges in computational modeling of affective processes. IEEE Trans. Affect. Comput. 4(3), 242–245 (2013). https://doi.org/10.1109/T-AFFC.2013.23

    Article  Google Scholar 

  22. Taverner, J., Vivancos, E., Botti, V.: A fuzzy appraisal model for affective agents adapted to cultural environments using the pleasure and arousal dimensions. Inf. Sci. 546, 74–86 (2021). https://doi.org/10.1016/j.ins.2020.08.006

    Article  Google Scholar 

  23. Ashwin, T.S., Guddeti, R.M.R.: Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks. Educ. Inf. Technol. 25(2), 1387–1415 (2019). https://doi.org/10.1007/s10639-019-10004-6

    Article  Google Scholar 

  24. Bird, J.J., Ekárt, A., Faria, D.R.: Chatbot interaction with artificial intelligence: human data augmentation with T5 and language transformer ensemble for text classification. ArXiv: 2010.05990 [Cs] (2020). http://arxiv.org/abs/2010.05990

  25. Fiorini, L., Mancioppi, G., Semeraro, F., Fujita, H., Cavallo, F.: Unsupervised emotional state classification through physiological parameters for social robotics applications. Knowl.-Based Syst. 190, 105217 (2020). https://doi.org/10.1016/j.knosys.2019.105217

  26. Ivanovic, M., et al.: Emotional agents—state of the art and applications. Comput. Sci. Inf. Syst. (2015). https://doi.org/10.2298/CSIS141026047I

  27. Newell, A.: SOAR as a unified theory of cognition: issues and explanations. Behav. Brain Sci. 15(3), 464–492 (1992). https://doi.org/10.1017/S0140525X00069740

    Article  Google Scholar 

  28. Zuo, G., Pan, T., Zhang, T., Yang, Y.: SOAR improved artificial neural network for multistep decision-making tasks. Cogn. Comput. 13(3), 612–625 (2020). https://doi.org/10.1007/s12559-020-09716-6

    Article  Google Scholar 

  29. Schindler, S., Bublatzky, F.: Attention and emotion: an integrative review of emotional face processing as a function of attention. Cortex 130, 362–386 (2020). https://doi.org/10.1016/j.cortex.2020.06.010

    Article  Google Scholar 

  30. Marcos, S., Gómez-García-Bermejo, J., Zalama, E.: A realistic, virtual head for human–computer interaction. Interact. Comput. 22(3), 176–192 (2010). https://doi.org/10.1016/j.intcom.2009.12.002

    Article  Google Scholar 

  31. Fernandez, R., John, N., Kirmani, S., Hart, J., Sinapov, J., Stone, P.: Passive demonstrations of light-based robot signals for improved human interpretability. In: 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 234–239. https://doi.org/10.1109/ROMAN.2018.8525728

  32. MacDorman, K.F., Ishiguro, H.: The uncanny advantage of using androids in cognitive and social science research. Interact. Stud.: Soc. Behav. Commun. Biol. Artif. Syst. 7(3), 297–337 (2006). https://doi.org/10.1075/is.7.3.03mac

    Article  Google Scholar 

  33. Matthews, G., et al.: Evolution and revolution: personality research for the coming world of robots, artificial intelligence, and autonomous systems. Pers. Individ. Differ. 169, 109969 (2021). https://doi.org/10.1016/j.paid.2020.109969

  34. Korn, O., Akalin, N., Gouveia, R.: Understanding cultural preferences for social robots: a study in German and Arab communities. ACM Trans. Hum.-Robot Interact. 10(2), 12:1–12:19 (2021). https://doi.org/10.1145/3439717

  35. Nishio, T., Yoshikawa, Y., Ogawa, K., Ishiguro, H.: Development of an effective information media using two android robots. Appl. Sci. 9(17), 3442 (2019). https://doi.org/10.3390/app9173442

    Article  Google Scholar 

  36. Doering, M., Glas, D.F., Ishiguro, H.: Modeling interaction structure for robot imitation learning of human social behavior. IEEE Trans. Hum.-Mach. Syst. 49(3), 219–231 (2019). https://doi.org/10.1109/THMS.2019.2895753

    Article  Google Scholar 

  37. Pablos, S.M., García-Bermejo, J.G., Zalama Casanova, E., López, J.: Dynamic facial emotion recognition oriented to HCI applications. Interact. Comput. 27(2), 99–119 (2015). https://doi.org/10.1093/iwc/iwt057

    Article  Google Scholar 

  38. Strathearn, C., Ma, M.: Modelling user preference for embodied artificial intelligence and appearance in realistic humanoid robots. Informatics 7(3), 28 (2020). https://doi.org/10.3390/informatics7030028

    Article  Google Scholar 

  39. Cañamero, L.: Emotion understanding from the perspective of autonomous robots research. Neural Netw. 18(4), 445–455 (2005). https://doi.org/10.1016/j.neunet.2005.03.003

    Article  Google Scholar 

  40. Dautenhahn, K., Woods, S., Kaouri, C., Walters, M. L., Koay, K.L., Werry, I.: What is a robot companion—Friend, assistant or butler? In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1192–1197 (2005). https://doi.org/10.1109/IROS.2005.1545189

  41. Whittaker, S., Rogers, Y., Petrovskaya, E., Zhuang, H.: Designing Personas for expressive robots: personality in the new breed of moving, speaking, and colorful social home robots. ACM Trans. Hum.-Robot Interact. 10(1), 8:1–8:25 (2021). https://doi.org/10.1145/3424153

  42. Schoenherr, J.R., Burleigh, T.J.: Uncanny sociocultural categories. Front. Psychol. 5. (2015). https://doi.org/10.3389/fpsyg.2014.01456

  43. Cheetham, M.: Editorial: the uncanny valley hypothesis and beyond. Front. Psychol. 8 (2017). https://doi.org/10.3389/fpsyg.2017.01738

  44. Gee, F.C., Browne, W.N., Kawamura, K.: Uncanny valley revisited. In: ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, pp. 151–157 (2005). https://doi.org/10.1109/ROMAN.2005.1513772

  45. Brink, K.A., Gray, K., Wellman, H.M.: Creepiness creeps In: uncanny valley feelings are acquired in childhood. Child Dev. 90(4), 1202–1214. (2019). https://doi.org/10.1111/cdev.12999

  46. Feng, S., et al.: The uncanny valley effect in typically developing children and its absence in children with autism spectrum disorders. PLoS One 13(11), e0206343 (2018). https://doi.org/10.1371/journal.pone.0206343

  47. Tinwell, A., Sloan, R.J.S.: Children’s perception of uncanny human-like virtual characters. Comput. Hum. Behav. 36, 286–296 (2014). https://doi.org/10.1016/j.chb.2014.03.073

    Article  Google Scholar 

Download references

Acknowledgments

This research was partially funded by the Spanish Government Ministry of Economy and Competitiveness through the DEFINES project grant number (TIN2016-80172-R) and the Ministry of Science and Innovation through the AVisSA project grant number (PID2020-118345RB-I00).

Author information

Authors and Affiliations

  1. GRIAL Research Group, University of Salamanca, Salamanca, Spain

    Samuel Marcos-Pablos & Francisco José García-Peñalvo

Authors
  1. Samuel Marcos-Pablos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco José García-Peñalvo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samuel Marcos-Pablos .

Editor information

Editors and Affiliations

  1. Faculty of Science, University of Salamanca, Salamanca, Spain

    Juan F. de Paz Santana

  2. Facultad de Informática, Universidad Pontificia de Salamanca, Salamanca, Spain

    Daniel H. de la Iglesia

  3. Facultad de Informática, Pontifical University of Salamanca, Salamanca, Spain

    Alfonso José López Rivero

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Marcos-Pablos, S., García-Peñalvo, F.J. (2022). Emotional Intelligence in Robotics: A Scoping Review. In: de Paz Santana, J.F., de la Iglesia, D.H., López Rivero, A.J. (eds) New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence. DiTTEt 2021. Advances in Intelligent Systems and Computing, vol 1410. Springer, Cham. https://doi.org/10.1007/978-3-030-87687-6_7

Download citation

Publish with us

Policies and ethics

Search

Navigation