Abstract
Emotion recognition is a field of study that is much investigated by researchers because it is an important piece in the development of intelligent system. Researchers used different techniques to capture the emotional response of the people based on the situation and design patterns to see its effects and relevance. In this paper, the researchers investigated the emotional patterns of the students towards programming using input devices such as keyboard and mouse. The use of keyboard and mouse provided an affordable and non-intrusive method of collecting data. The keyboard and mouse data collected were mapped by an expert on facial expressions to emotions captured on video at the same the students were answering the programming problem. From these annotations, the classes of emotions were derived. Rapid miner was then used to identify the pattern of keyboard and mouse strokes that corresponds to different emotions. The accuracy of the results were 70.25% and with a Kappa of 0.612%.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gross, J.J., Muñoz, R.F.: Emotion regulation and mental health. Clin. Psychol. Sci. Pract. 2(2), 151–164 (1995)
Salovey, P., Rothman, A.J., Detweiler, J.B., Steward, W.T.: Emotional states and physical health. Am. Psychol. 55(1), 110 (2000)
Picard, R.W.: Affective computing: from laughter to IEEE. IEEE Trans. Affect. Comput. 1(1), 11–17 (2010)
Picard, R.W., Picard, R.: Affective Computing, vol. 252. MIT press, Cambridge (1997)
Khanna, P., Sasikumar, M.: Recognizing emotions from keyboard stroke pattern. Int. J. Comput. Appl
Khan, I.A., Brinkman, W.P., Hierons, R.: Towards estimating computer users’ mood from interaction behaviour with key board and mouse. Front. Comput. Sci. 7(6), 943–954 (2013)
Salmeron-Majadas, S., Santos, O.C., Boticario, J.G.: Exploring indicators from keyboard and mouse interactions to predict the user affective state. Educ. Data Min. (2014)
Grimes, G.M., Jenkins, J.L., Valacich, J.S.: Exploring the effect of arousal and valence on mouse interaction. In: 34th International Conference on Information Systems. Milan (2013)
Lang, P., Bradley, M.M.: The International Affective Picture System (IAPS) in the Study of Emotion and Attention, Handbook of emotion elicitation and assessment, p. 29 (2007)
Lang, P.J.: The emotion probe. Am. Psychol. 50(5), 372–385 (1995)
Bradley, M.M., Greenwald, M.K., Petry, M.C., Lang, P.J.: Remembering pictures: pleasure and arousal in memory. J. Exp. Psychol. Learn. Mem. Cognit. 18(2), 379–390 (1992)
Greenwald, M.K., Bradley, M.M., Hamm, A.O., Lang, P.J.: Looking at pictures: evaluative, facial, visceral and behavioral responses. Psychophysiology 30(3), 261–273 (1993)
Miller, G.A., Levin, D.N., Kozak, M.J., Cook III, E.W., McLean Jr., A., Lang, P.J.: Individual differences in imagery and the psychophysiology of emotion. Cognit. Emot. 1(4), 367–390 (1987)
Salmeron-Majadas, S., Santos, O.C., Boticario, J.G.: An evaluation of mouse and keyboard interaction indicators towards non-intrusive and low cost affective modeling in an educational context. Procedia Comput. Sci. 35, 691–700 (2014)
Ball, G., Breese, J.: Emotion and Personality in a Conversational Character. Embodied Conversational Agents. MIT Press, Cambridge (2000)
Bixler, R., D’ Mello, S.: Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits. In: Proceedings of the 2013 International Conference on Intelligent User Interfaces, pp. 225–234. ACM, March 2013
Zimmermann, P., Guttormsen, S., Danuser, B., Gomez, P.: Affective computing—a rationale for measuring mood with mouse and keyboard. Int. J. Occup. Safety Ergon. 9(4), 539–551 (2003)
Rodrigues, M., Gonçalves, S., Carneiro, D., Novais, P., Fdez-Riverola, F.: Keystrokes and clicks: measuring stress on e-learning students. In: Casillas, J., MartÃnez-López, F., Vicari, R., De la Prieta, F. (eds.) Management Intelligent Systems. Advances in Intelligent Systems and Computing, vol. 220, pp. 119–126. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-319-00569-0_15
Shukla, P., Solanki, R.: Web based keystroke dynamics application for identifying emotional state. Institute of Engineering and Technology DAVV, Indore, India, November 2013
Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., Wróbel, M.R.: Emotion recognition and its application in software engineering. In: 2013 The 6th International Conference on Human System Interaction (HSI), pp. 532–539. IEEE, June 2013
Lee, P.M., Tsui, W.H., Hsiao, T.C.: The influence of emotion on keyboard typing: an experimental study using auditory stimuli. PLoS ONE 10(6), e0129056 (2015)
Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. stat. 29(5), 1189–1232 (2001)
Mitra, S., Pal, S.K., Mitra, P.: Data mining in soft computing framework: a survey. IEEE Trans. Neural Netw. 13(1), 3–14 (2002)
Click, C., Malohlava, M., Candel, A., Roark, H., Parmar, V.: Gradient Boosted Models with H2O, March 2016. https://h2o-release.s3.amazonaws.com/h2o/rel-turan/4/docs-website/h2o-docs/booklets/GBM_Vignette.pdf.Amazonaws.com. Accessed M ay 2017
Cao, G., Ding, J., Duan, Y., Tu, L., Xu, J., Xu, D.: Classification of tongue images based on doublet and color space dictionary. In: IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1170–1175 (2016)
Mchugh, M.L.: Interrater reliability: the kappa statistic. Biochemia Medica, pp. 276–282 (2012)
Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Therapy Exp. Psychiatry 25(1), 49–59 (1994)
Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147, 195–197 (1981)
May, P., Ehrlich, H.C., Steinke, T.: ZIB structure prediction pipeline: composing a complex biological workflow through web services. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.) Euro-Par 2006. LNCS, vol. 4128, pp. 1148–1158. Springer, Heidelberg (2006)
Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (1999)
Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for distributed resource sharing. In: 10th IEEE International Symposium on High Performance Distributed Computing, pp. 181–184. IEEE Press, New York (2001)
Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The physiology of the grid: an open grid services architecture for distributed systems integration. Technical report, Global Grid Forum (2002)
National Center for Biotechnology Information. http://www.ncbi.nlm.nih.gov
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Estrada, J., Buhia, J., Guevarra, A., Forcado, M.R. (2018). Keyboard and Mouse: Tools in Identifying Emotions During Computer Activities. In: Jung, J., Kim, P., Choi, K. (eds) Big Data Technologies and Applications. BDTA 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 248. Springer, Cham. https://doi.org/10.1007/978-3-319-98752-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-98752-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98751-4
Online ISBN: 978-3-319-98752-1
eBook Packages: Computer ScienceComputer Science (R0)