Abstract
In an attempt to predict the cognitive-affective states of a player during an educational video game session, this study used a self-emote procedure in which participants’ facial expressions and emotions were continuously recorded along with self-reported data about their emotional states. Participants’ facial expressions and emotions were captured using Affdex SDK from Affectiva. The captured data were used for binomial logistic regression to predict the cognitive-affective states of flow, frustration, and boredom. The binomial logistic regression uncovered that expressions and emotions could be used to predict these cognitive-affective states of a player. We discuss these predictors and their potential to adapt an educational video game session with non-intrusive and affect-sensitive personalization capabilities. The current study provides a pathway for the educational play design and suggests that it should be non-intrusive while being adaptive to a player’s capabilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baron, T.: An Architecture for designing content agnostic game mechanics for educational burst games. Ph.D. thesis, Arizona State University (2017)
Baron, T., Amresh, A.: Word towers: assessing domain knowledge with non-traditional genres. In: European Conference on Games Based Learning, p. 638. Academic Conferences International Limited (2015)
Bosch, N., Chen, H., D’Mello, S., Baker, R., Shute, V.: Accuracy vs. availability heuristic in multimodal affect detection in the wild. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 267–274 (2015)
Craig, S., Graesser, A., Sullins, J., Gholson, B.: Affect and learning: an exploratory look into the role of affect in learning with autotutor. J. Educ. Media 29(3), 241–250 (2004)
Craig, S.D., D’Mello, S., Witherspoon, A., Graesser, A.: Emote aloud during learning with autotutor: applying the facial action coding system to cognitive-affective states during learning. Cogn. Emot. 22(5), 777–788 (2008)
Czikszentmihalyi, M.: Flow: The Psychology of Optimal Experience (1990)
D’Mello, S., Graesser, A.: Emotions during learning with autotutor. Adapti. Technol. Training Educ. pp. 169–187 (2012)
D’Mello, S., Kappas, A., Gratch, J.: The affective computing approach to affect measurement. Emot. Rev. 10(2), 174–183 (2018)
Dmello, S.K., Craig, S.D., Witherspoon, A., Mcdaniel, B., Graesser, A.: Automatic detection of learner’s affect from conversational cues. User Model. User-Adapt. Inter. 18(1–2), 45–80 (2008). https://doi.org/10.1007/s11257-007-9037-6
Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement, Psychologists press. Palo Alto (1978)
Harley, J.M.: Measuring emotions: a survey of cutting edge methodologies used in computer-based learning environment research. In: Emotions, Technology, Design, and Learning, pp. 89–114. Elsevier (2016)
iMotions Inc.: Affectiva channel explained (2018). https://help.imotions.com/hc/en-us/articles/360011728719-Affectiva-channel-explained. Accessed 17 Apr 2020
Magdin, M., Prikler, F.: Real time facial expression recognition using webcam and sdk affectiva. IJIMAI 5(1), 7–15 (2018)
Sabourin, J., Mott, B., Lester, J.C.: Modeling learner affect with theoretically grounded dynamic bayesian networks. In: DMello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6974, pp. 286–295. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_32
Tadayon, R., Amresh, A., McDaniel, T., Panchanathan, S.: Real-time stealth intervention for motor learning using player flow-state. In: 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH), pp. 1–8. IEEE (2018)
Yun, C., Shastri, D., Pavlidis, I., Deng, Z.: O’game, can you feel my frustration? improving user’s gaming experience via stresscam. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2195–2204 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Verma, V., Rheem, H., Amresh, A., Craig, S.D., Bansal, A. (2020). Predicting Real-Time Affective States by Modeling Facial Emotions Captured During Educational Video Game Play. In: Marfisi-Schottman, I., Bellotti, F., Hamon, L., Klemke, R. (eds) Games and Learning Alliance. GALA 2020. Lecture Notes in Computer Science(), vol 12517. Springer, Cham. https://doi.org/10.1007/978-3-030-63464-3_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-63464-3_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63463-6
Online ISBN: 978-3-030-63464-3
eBook Packages: Computer ScienceComputer Science (R0)