Skip to main content
SpringerLink
Log in

In search of a measure to address different sources of cognitive load in computer-based learning environments

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

In the current study, we aimed to develop a reliable and valid scale to address individual cognitive load types. Existing scale development studies involved limited number of items without adequate convergent, discriminant and criterion validity checks. Through a multistep correlational study, we proposed a three-factor scale with 13 items to address intrinsic, extraneous and germane cognitive load in computer-based learning environments. A thorough literature search of cognitive load indicators in the literature was followed by expert panels for content and construct validity. The initial item pool was administered to 236 undergraduate students who watched an instructional video on IP address classes. A multiple-choice achievement test was also implemented after the intervention. The exploratory factor analysis with maximum likelihood explained 58 percent of the variance with factor loads above .49, and internal consistency coefficients above .81. Convergent and discriminant validity indices were acceptable. Besides, achievement was related positively with the germane load and negatively with intrinsic and extraneous load. Fifteen percent of the achievement was explained by the three sources of the cognitive load. Then, the developed factor structure was validated with 193 undergraduate students immediately after they participated in online webinars, and with 99 undergraduate students after they participated in face to face classes. The proposed structure was confirmed in both settings so the scale was considered a reliable and valid indicator of cognitive load in both online and face-to-face learning environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Albus, P., Vogt, A., & Seufert, T. (2021). Signaling in virtual reality influences learning outcome and cognitive load. Computers & Education, 166, 104154.

    Article  Google Scholar 

  • Anmarkrud, Ø., Andresen, A., & Bråten, I. (2019). Cognitive load and working memory in multimedia learning: Conceptual and measurement issues. Educational Psychologist, 54(2), 61–83.

    Article  Google Scholar 

  • Ayres, P. (2006). Using subjective measures to detect variations of intrinsic cognitive load within problems. Learning and Instruction, 16(5), 389–400.

    Article  Google Scholar 

  • Ayres, P. (2017). Subjective measures of cognitive load: What can they reliability measure? In R. Z. Zheng (Ed.), Cognitive load measurement and application: A theoretical framework for meaningful research and practice (pp. 9–28). Routledge.

    Google Scholar 

  • Ayres, P. (2018). Subjective measures of cognitive load: What can they reliably measure? In R. Z. Zheng (Ed.), Cognitive load measurement and application: A theoretical framework for meaningful research and practice (pp. 9–28). Routledge/Taylor & Francis Group

  • Ayres, P. (2020). Something old, something new from cognitive load theory. Computers in Human Behavior, 113, 106503.

    Article  Google Scholar 

  • Baddeley, A. (1996). Exploring the central executive. The Quarterly Journal of Experimental Psychology Section A, 49(1), 5–28.

    Article  Google Scholar 

  • Bender, L., Renkl, A., & Eitel, A. (2021). Seductive details do their damage also in longer learning sessions–When the details are perceived as relevant. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12560

    Article  Google Scholar 

  • Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149.

    Article  Google Scholar 

  • Brown, T. A. (2006). Confirmatory factor analysis for applied research. Guilfor Publications.

    Google Scholar 

  • Brunken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38(1), 53–61.

    Article  Google Scholar 

  • Brunken, R., Seufert, T., & Paas, F. (2010). Measuring cognitive load. In J. L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory (pp. 181–202). Cambridge University Press.

    Chapter  Google Scholar 

  • Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in Human Behavior, 25(2), 315–324.

    Article  Google Scholar 

  • Cook, A., Zheng, R., & Blaz, J. W. (2009). Measurement of cognitive load during multimedia learning activities. In R. Zheng (Ed.), Cognitive effectives of multimedia learning (pp. 34–50). Information Science Reference/IGI Global Publishing.

    Chapter  Google Scholar 

  • Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), 7.

    Google Scholar 

  • Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87–114.

    Article  Google Scholar 

  • DeVellis, R. F. (2017). Scale Development: Theory and Applications (4th ed.). Sage Publications.

    Google Scholar 

  • Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299.

    Article  Google Scholar 

  • Fisher, R. J. (2000). The future of social-desirability bias research in marketing. Psychology & Marketing, 17(2), 73–77.

    Article  MathSciNet  Google Scholar 

  • Fokkema, M., & Greiff, S. (2017). How performing PCA and CFA on the same data equals trouble. European Journal of Psychological Assessment, 33(6), 399–402.

    Article  Google Scholar 

  • Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50.

    Article  Google Scholar 

  • George, D., & Mallery, P. (2010). SPSS for Windows step by step. A simple study guide and reference (10th ed.). Allyn and Bacon.

    Google Scholar 

  • Hair, J. F., Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2016). A primer on partial least squares structural equation modeling (PLS-SEM). Sage publications.

    MATH  Google Scholar 

  • Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human Mental Workload. North Holland Press.

    Google Scholar 

  • Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research. Educational and Psychological Measurement, 66, 393–416. https://doi.org/10.1177/0013164405282485

    Article  MathSciNet  Google Scholar 

  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

    Article  Google Scholar 

  • Hutcheson, G., & Sofroniou, N. (1999). The multivariate social scientist. Sage.

    Book  MATH  Google Scholar 

  • Johannessen, E., Szulewski, A., Radulovic, N., White, M., Braund, H., Howes, D., ... & Davies, C. (2020). Psychophysiologic measures of cognitive load in physician team leaders during trauma resuscitation. Computers in Human Behavior, 111, 106393.

  • Kalyuga, S. (2010). Schema acquisition and sources of cognitive load. In J. L. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive load theory (pp. 48–64). Cambridge University Press.

    Chapter  Google Scholar 

  • Kalyuga, S. (2011). Cognitive load theory: How many types of load does it really need? Educational Psychology Review, 23(1), 1–19.

    Article  Google Scholar 

  • Kirschner, P. A., Ayres, P., & Chandler, P. (2011). Contemporary cognitive load theory research: The good, the bad and the ugly. Computers in Human Behavior, 27(1), 99–105.

    Article  Google Scholar 

  • Kışla, T., Emirtekin, E., Polan, Ş., & Dönmez, O. (2020). Etkileşimli eğitsel video ve başarı testinin geliştirilmesi: IP adresi kavramı örneği [Development of Interactive Educational Video and Achievement Test: A Case of IP Address Concept]. Journal of Instructional Technologies and Teacher Education, 9(1), 42–51.

  • Klepsch, M., Schmitz, F., & Seufert, T. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in Psychology, 8, 1997.

    Article  Google Scholar 

  • Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). Guilford Press.

    MATH  Google Scholar 

  • Lee, J. Y., Donkers, J., Jarodzka, H., Sellenraad, G., & Van Merriënboer, J. J. (2020). Different effects of pausing on cognitive load in a medical simulation game. Computers in Human Behavior, 110, 106385.

    Article  Google Scholar 

  • Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45, 1058–1072.

    Article  Google Scholar 

  • Moreno, R., & Park, B. (2010). Cognitive load theory: Historical development and relation to other theories. In J. L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory (pp. 9–28). Cambridge University Press.

    Chapter  Google Scholar 

  • Morrison, B. B., Dorn, B., & Guzdial, M. (2014). Measuring cognitive load in introductory CS: adaptation of an instrument. In Proceedings of the tenth annual conference on International computing education research (pp. 131–138).

  • Mundfrom, D. J., Shaw, D. G., & Ke, T. L. (2005). Minimum sample size recommendations for conducting factor analyses. International Journal of Testing, 5(2), 159–168.

    Article  Google Scholar 

  • Mutlu-Bayraktar, D., Cosgun, V., & Altan, T. (2019). Cognitive load in multimedia learning environments: A systematic review. Computers & Education, 141, 103618.

    Article  Google Scholar 

  • Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGrawHill.

    Google Scholar 

  • Örün, Ö., & Akbulut, Y. (2019). Effect of multitasking, physical environment and electroencephalography use on cognitive load and retention. Computers in Human Behavior, 92, 216–229.

  • Paas, F. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429.

    Article  Google Scholar 

  • Paas, F., & van Merriënboer, J. J. (2020). Cognitive-load theory: Methods to manage working memory load in the learning of complex tasks. Current Directions in Psychological Science, 29(4), 394–398.

    Article  Google Scholar 

  • Paas, F. G., Van Merriënboer, J. J., & Adam, J. J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79(1), 419–430.

    Article  Google Scholar 

  • Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4.

    Article  Google Scholar 

  • Schneider, S., Krieglstein, F., Beege, M., & Rey, G. D. (2022). The impact of video lecturers’ nonverbal communication on learning–An experiment on gestures and facial expressions of pedagogical agents. Computers & Education, 176, 104350.

    Article  Google Scholar 

  • Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19(4), 469–508.

    Article  Google Scholar 

  • Schumacker, R. E., & Lomax, R. G. (1996). A beginner’s guide to structural equation modeling. Lawrence Erlbaum Associates.

    MATH  Google Scholar 

  • Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898.

    Article  Google Scholar 

  • Sumer, N. (2000). Structural equation models. Turkish Psychological Articles, 3(6), 49–74.

    MathSciNet  Google Scholar 

  • Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.

    Article  Google Scholar 

  • Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138.

    Article  Google Scholar 

  • Sweller, J. (2018). Measuring cognitive load. Perspectives on Medical Education, 7(1), 1–2.

    Article  Google Scholar 

  • Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.

    Article  Google Scholar 

  • Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer.

    Book  Google Scholar 

  • Sweller, J., van Merriënboer, J. J., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292.

    Article  Google Scholar 

  • Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Allyn & Bacon.

    Google Scholar 

  • Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson.

    Google Scholar 

  • Thees, M., Kapp, S., Strzys, M. P., Beil, F., Lukowicz, P., & Kuhn, J. (2020). Effects of augmented reality on learning and cognitive load in university physics laboratory courses. Computers in Human Behavior, 108, 106316.

    Article  Google Scholar 

  • Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association.

    Book  Google Scholar 

  • Van Merrienboer, J. J., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177.

    Article  Google Scholar 

  • Van Peppen, L. M., Verkoeijen, P. P., Heijltjes, A. E., Janssen, E. M., & van Gog, T. (2021). Enhancing students’ critical thinking skills: Is comparing correct and erroneous examples beneficial? Instructional Science, 49, 747–777.

    Article  Google Scholar 

  • Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806–838.

    Article  Google Scholar 

  • Zheng, R. Z. (2018). Cognitive Load Measurement and Application: A Theoretical Framework for Meaningful Research and Practice. Routledge.

    Google Scholar 

Download references

Acknowledgements

We thank Editors and anonymous reviewers for their outstanding feedback. We also thank hundreds of undergraduate students who contributed to the current research through their invaluable responses.

Author information

Authors and Affiliations

  1. Department of Computer Education & Instructional Technology, Faculty of Education, Ege University, İzmir, Turkey

    Onur Dönmez

  2. Department of Educational Sciences, Faculty of Education, Anadolu University, Eskişehir, Turkey

    Yavuz Akbulut

  3. Department of Computer Education & Instructional Technology, Faculty of Education, Erzincan Binali Yıldırım University, Ankara, Turkey

    Esra Telli

  4. Department of Computer Education & Instructional Technology, Graduate School of Educational Sciences, Ege University, İzmir, Turkey

    Miray Kaptan & İbrahim H. Özdemir

  5. Department of Computer Education & Instructional Technology, Faculty of Education, Hacettepe University, Ankara, Turkey

    Mukaddes Erdem

Authors
  1. Onur Dönmez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yavuz Akbulut
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Esra Telli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miray Kaptan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. İbrahim H. Özdemir
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mukaddes Erdem
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Onur Dönmez: Conceptualization, Methodology, Software development, Data collection, Writing—original draft. Yavuz Akbulut: Conceptualization, Methodology, Formal Analysis, Writing—original draft. Esra Telli, Miray Kaptan, İbrahim H. Özdemir, Mukaddes Erdem: Conceptualization, Methodology, Writing—original draft.

Corresponding author

Correspondence to Onur Dönmez.

Ethics declarations

Research ethics statement

This study was approved by the Ege University Research Ethics Committee (approval no. 06/26–926). All participants provided written informed consent prior to enrolment in the study.

Conflict of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dönmez, O., Akbulut, Y., Telli, E. et al. In search of a measure to address different sources of cognitive load in computer-based learning environments. Educ Inf Technol 27, 10013–10034 (2022). https://doi.org/10.1007/s10639-022-11035-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-022-11035-2

Keywords

Search

Navigation