Abstract
This study investigated differences in Chinese as a Foreign Language (CFL) students’ learning achievement, cognitive load level, and eye-movement patterns when learning three types of Chinese characters: pictographs, compound ideographs, and phono-semantic compounds with multimedia slides. Thirty CFL students participated in this study. Additional eye-tracking data, such as participants’ total fixation duration, mean fixation duration, and fixation count while viewing instructional slides, were recorded. The results showed that CFL learners learned pictograph characters with less cognitive load, compared with that involved in learning compound ideograph and phono-semantic compound characters. Participants were found to fixate more on the character area when inspecting the pictograph than on the other two categories of characters. Meanwhile, fewer fixations were placed on the enlarged character area when text passages served as the major resource for understanding compound ideograph and phono-semantic compound characters. The illustrative nature of pictograph characters seemed to make this character form more explicit than the other categories of Chinese characters. However, using only text information to understand abstract meanings of compound ideographs and phono-semantic compounds may place an additional cognitive demand on CFL learners, resulting in learners spending less time inspecting the character shape. Replacing the text description with videos or animation may provide scenarios in which ambiguity in the meaning of compound ideographs and phono-semantic compounds can be removed, to promote better understanding of these characters for CFL learners. Future studies are recommended to determine if the use of voice-over and videos/animation explanations improve the ability to learn Chinese characters.
Similar content being viewed by others
Reference
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.
Beebe, L. (1985). Input: Choosing the right stuff. In S. Gass & C. Madden (Eds.), Input in second language acquisition (pp. 404–414). Rowley, MA: Newbury House.
Block, D. (2003). The social turn in second language acquisition. Washington, D.C.: George-town University Press.
Chen, H.-Y., & Liu, K.-Y. (2008). Web-based synchronized multimedia lecture system design for teaching/learning Chinese as second language. Computers & Education, 50(3), 693–702. https://doi.org/10.1016/j.compedu.2006.07.010.
Chinese character classification. (2018). In Wikipedia. Retrieved November 13, 2018, from https://en.wikipedia.org/wiki/Chinese_character_classification.
Chuang, H.-Y., & Ku, H.-Y. (2011). The effect of computer-based multimedia instruction with Chinese character recognition. Educational Media International, 48(1), 27–41. https://doi.org/10.1080/09523987.2011.549676.
Cook, V. (2003). Introduction: The changing LI in the L2 user's mind. In V. Cook (Ed.), The effects of learning a second language on the first: The case of increased metalinguistic awareness (pp. 1–18). Clevedon: Multilingual Matters.
de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2010). Attention guidance in learning from a complex animation: Seeing is understanding? Learning and Instruction, 20(2), 111–122.
Du, J., Zhai, J.F., Hu, J.S., Zhu, B., Wei, S., & Dai, L.R. (2015). Writer adaptive feature extraction based on convolutional neural networks for online handwritten Chinese character recognition. Paper presented at 13th international conference in Tubis.
Glaholt, M. G., Wu, M.-C., & Reingold, E. M. (2010). Evidence for top-down control of eye movements during visual decision making. Journal of Vision, 10, 1–10.
Hegarty, M. (1992). The mechanics of comprehension and comprehension of mechanics. In K. Rayner (Ed.), Eye movements and visual cognition. Springer series in neuropsychology (pp. 428–443). New York: Springer.
Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742.
Hyönä, J. (2010). The use of eye movements in the study of multimedia learning. Learning and Instruction, 20(2), 172–176.
Jian, Y-C., Chen, M. L., & Ko, H. W. (2013). Context effects in processing of Chinese academic words: An eye-tracking investigation. Reading Research Quarterly, 48(4), 403–413. https://doi.org/10.1002/rrq.56.
Jiang, X. (2008). Research on words of teaching Chinese as a second language and learning to read. Beijing: Beijing Language and Culture University Press.
Kikas, E. (2006). The effect of verbal and visuo-spatial abilities on the development of knowledge of the Earth. Research in Science Education, 36(3), 269.
Krashen, S. D. (1985). The input hypothesis: Issues and implications. London: Longman.
Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-Computer Studies, 65(11), 911–930.
Kuo, M-L., & Hooper, S. (2004). The effects of visual and verbal coding mnemonics on learning Chinese characters in computer-based instruction. Educational Technology Research and Development, 52, 23–34. https://doi.org/10.1007/BF02504673.
Lee, C.-P., Shen, C.-W., & Lee, D. (2008). The effect of multimedia instruction for Chinese learning. Learning, Media and Technology, 33(2), 127–138. https://doi.org/10.1080/17439880802097667.
Mason, A., & Zhang, W. (2017). An exploration of the use of mobile applications to support the learning of Chinese characters employed by students of Chinese as a foreign language. https://doi.org/10.14705/rpnet.2017.mooc2016.674
Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.
Ozcelik, E., Karakus, T., Kursun, E., & Cagiltay, K. (2009). An eye-tracking study of how color coding affects multimedia learning. Computers & Education, 53(2), 445–453.
Paas, F. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429–434
Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86(1), 122–133.
Rayner, K. (1990). Eye movements and visual cognition: Introduction (pp. 1–7). New York: Springer-Verlag New York Inc.
Rayner, K. (1998). Eye movements in reading and information processing. 20 years of research. Psychological Bulletin, 124(3), 372–422.
Rayner, K., Abbott, M., Schotter, E., Belanger, N., Higgins, E., Leinenger, M., et al. (2015). Keith Rayner eye movements in reading data collection. UC San Diego Library Digital Collections. https://doi.org/10.6075/J0JW8BSV.
Renshaw, J. A., Finlay, J. E., Tyfa, D., & Ward, R. D. (2004). Understanding visual influence in graph design through temporal and spatial eye movement characteristics. Interacting with Computers, 16, 557–578.
Rites of Zhou. (2018). In Wikipedia. Retrieved December 10, 2018, from https://en.wikipedia.org/wiki/Rites_of_Zhou
Shen, H. (2004). Level of cognitive processing: Effects on character learning among non-native learners of Chinese as a foreign language. Language and Education, 18(2), 167–182.
Shi, D., Damper, R. I., & Gunn, S. R. (2003). An approach to off-line handwritten Chinese character recognition based on hierarchical radical decomposition. Journal of Quantitative Linguistics, 10(1), 41–69.
Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19, 469–508. https://doi.org/10.1007/s10648-007-9053-4.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312.
Underwood, G., & Radach, R. (1998). Eye guidance and visual information processing: Reading, visual search, picture perception and driving. In G. Underwood (Ed.), Eye guidance in reading and scene perception (pp. 1–27). Oxford: Elsevier.
van Gog, T., & Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 20(2), 95–99.
Wen, Y. (2018). Chinese character composition game with the augment paper. Educational Technology & Society, 21(3), 132–145.
Wieger, L. S. J. (1965). Chinese characters: Their origin, etymology, history, classification and signification. A thorough study from Chinese documents 2nd ed. English and revision according to the 4th French ed. New York: Paragon Book Reprint Corp.
Ye, L.-J. (2011). Teaching and learning Chinese as a foreign language in the United States: To delay or not to delay the character introduction (Doctoral dissertations, College of Art and Sciences of Georgia State University. Retrieved November 17, 2018, from https://scholarworks.gsu.edu/cgi/viewcontent.cgi?article=1021&context=alesl_diss
Zhen, Z. (2015). Advantages and thinking on design of Chinese characters’ graphics. Studies in Literature and Language, 11(2), 82–87. https://doi.org/10.3968/7447.
Zhou, P., Su, Y. E, Crain, S., Gao, L., & Zhan, L. (2012). Children’s use of phonological information in ambiguity resolution: A review from Mandarin Chinese. Journal of Child Language, 39, 687–730.
Acknowledgements
This research project was supported in part by the Ministry of Science and Technology, Taiwan; Grant No. MOST 106-2511-S-415-009-.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Liu, HC. Using Eye-Tracking Technology to Explore the Impact of Instructional Multimedia on CFL Learners’ Chinese Character Recognition. Asia-Pacific Edu Res 30, 33–46 (2021). https://doi.org/10.1007/s40299-020-00512-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40299-020-00512-2