Skip to main content
SpringerLink
Log in

iSTART Therefore I Understand: But Metacognitive Supports Did not Enhance Comprehension Gains

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10331))

Included in the following conference series:

Abstract

iSTART is an intelligent tutoring system designed to provide self-explanation instruction and practice to improve students’ comprehension of complex, challenging text. This study examined the effects of extended game-based practice within the system as well as the effects of two metacognitive supports implemented within this practice. High school students (n = 234) were either assigned to an iSTART treatment condition or a control condition. Within the iSTART condition, students were assigned to a 2 × 2 design in which students provided self-assessments of their performance or were transferred to Coached Practice if their performance did not reach a certain performance threshold. Those receiving iSTART training produced higher self-explanation and inference-based comprehension scores. However, there were no direct effects of either metacognitive support on these learning outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. NAEP: The Nation’s Report Card: Mathematics and Reading at Grade 12 (2015). https://www.nationsreportcard.gov/reading_math_g12_2015/#reading/acl#header

  2. McNamara, D.S.: SERT: self-explanation reading training. Discourse Process. 38, 1–30 (2004)

    Article  Google Scholar 

  3. McNamara, D.S., Levinstein, I.B., Boonthum, C.: iSTART: interactive strategy trainer for active reading and thinking. Behav. Res. Methods Instrum. Comput. 36, 222–233 (2004)

    Article  Google Scholar 

  4. Snow, E.L., Jacovina, M.E., Jackson, G.T., McNamara, D.S.: iSTART-2: a reading comprehension and strategy instruction tutor. In: McNamara, D.S., Crossley, S.A. (eds.) Adaptive Educational Technologies for Literacy Instruction, pp. 104–121. Taylor & Francis, Routledge (2016)

    Google Scholar 

  5. McNamara, D.S., Boonthum, C., Levinstein, I.B., Millis, K.: Evaluating self-explanations in iSTART: comparing word-based and LSA algorithms. In: Landauer, T., McNamara, D.S., Dennis, S., Kintsch, W. (eds.) Handbook of Latent Semantic Analysis, pp. 227–241. Erlbaum, Mahwah, NJ (2007)

    Google Scholar 

  6. Kintsch, W., Britton, B.K., Fletcher, C.R., Kintsch, E., Mannes, S.M., Nathan, M.J.: A comprehension-based approach to learning and understanding. Psychol. Learn. Motiv. Adv. Res. Theory 30, 165–214 (1993)

    Article  Google Scholar 

  7. Kurby, C.A., Magliano, J.P., Dandotkar, S., Woehrle, J., Gilliam, S., McNamara, D.S.: Changing how students process and comprehend texts with computer-based self-explanation training. J. Educ. Comput. Res. 47, 429–459 (2012)

    Article  Google Scholar 

  8. Magliano, J.P., Todaro, S., Millis, K., Wiemer-Hastings, K., Kim, H.J., McNamara, D.S.: Changes in reading strategies as a function of reading training: a comparison of live and computerized training. J. Educ. Comput. Res. 32, 185–208 (2005)

    Article  Google Scholar 

  9. O’Reilly T., Sinclair, G.P., McNamara, D.S.: iSTART: a web-based reading strategy intervention that improves students’ science comprehension. In: Kinshuk Sampson, D.G., Isaías, P. (eds.) Proceedings of the IADIS International Conference Cognition and Exploratory Learning in Digital Age: CELDA 2004. IADIS Press, Lisbon, Portugal, pp. 173–180 (2004)

    Google Scholar 

  10. Dickey, M.D.: Engaging by design: how engagement strategies in popular computer and video games can inform instructional design. Educ. Tech. Res. Dev. 53, 67–83 (2005)

    Article  Google Scholar 

  11. Jackson, G.T., McNamara, D.S.: Motivational impacts of a game-based intelligent tutoring system. In: Murray, R.C., McCarthy, P.M. (eds.) Proceedings of the 24th International Florida Artificial Intelligence Research Society (FLAIRS) Conference. AAAI Press, Menlo Park, CA, pp. 519–524 (2011)

    Google Scholar 

  12. Jackson, G.T., McNamara, D.S.: Motivation and performance in a game-based intelligent tutoring system. J. Educ. Psychol. 105, 1036–1049 (2013)

    Article  Google Scholar 

  13. Jackson, G.T., Snow, E.L., Varner, L.K., McNamara, D.S.: Game performance as a measure of comprehension and skill transfer. In Boonthum-Denecke C., Youngblood, G.M. (eds.) Proceedings of the 26th Annual Florida Artificial Intelligence Research Society (FLAIRS) Conference. The AAAI Press, Menlo Park, CA, pp. 497–502 (2013)

    Google Scholar 

  14. Theide, K.W., Dunlosky, J.: Toward a general model of self-regulated study: an analysis of selection of items and self-paced study. J. Exp. Psychol. Learn. Mem. Cogn. 4, 1024–1037 (1999)

    Article  Google Scholar 

  15. Azevedo, R.: Computer environments as metacognitive tools for enhancing learning. Educ. Psychol. 40, 193–197 (2005)

    Article  Google Scholar 

  16. Azevedo, R., Hadwin, A.F.: Scaffolding self-regulated learning and metacognition–Implications for the design of computer-based scaffolds. Instr. Sci. 33, 367–379 (2005)

    Article  Google Scholar 

  17. Roll, I., Aleven, V., McLaren, B.M., Koedinger, K.R.: Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learn. Instr. 21, 267–280 (2011)

    Article  Google Scholar 

  18. Schraw, G.: The effect of metacognitive knowledge on local and global monitoring. Contemp. Educ. Psychol. 19, 143–154 (1994)

    Article  Google Scholar 

  19. Snow, E.L., McNamara, D.S., Jacovina, M.E., Allen, L.K., Johnson, A.M., Perret, C.A., Dai, J., Tanner Jackson, G., Likens, A.D., Russell, D.G., Weston, J.L.: Promoting metacognitive awareness within a game-based intelligent tutoring system. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 786–789. Springer, Cham (2015). doi:10.1007/978-3-319-19773-9_116

    Chapter  Google Scholar 

  20. Snow, E.L., Jacovina, M.E., McNamara, D.S.: Promoting metacognition within a game-based environment. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 864–867. Springer, Cham (2015). doi:10.1007/978-3-319-19773-9_134

    Chapter  Google Scholar 

  21. Flavell, J.H.: Metacognition and cognitive monitoring: a new areas of cognitive-developmental inquiry. Am. Psychol. 34, 906–911 (1979)

    Article  Google Scholar 

  22. Roll, I., Aleven, V., McLaren, B.M., Koedinger, K.R.: Designing for metacognition – applying cognitive tutor principles to the tutoring of help seeking. Metacognition Learn. 2, 125–140 (2007)

    Article  Google Scholar 

  23. Healy, A.F., Clawson, D.M., McNamara, D.S., Marmie, W.R., Schneider, V.I., Rickard, T.C., Curtcher, R.J., King, C., Ericsson, K.A., Bourne, L.E.: The long term retention of knowledge and skills. Psychol. Learn. Motiv. Adv. Res. Theory 30, 135–164 (1993)

    Article  Google Scholar 

  24. Ericsson, K.A., Krampe, R.T., Tesch-Römer, C.: The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100, 363–406 (1993)

    Article  Google Scholar 

  25. McNamara, D.S., O’Reilly, T., Best, R., Ozuru, Y.: Improving adolescent students’ reading comprehension with iSTART. J. Educ. Comput. Res. 34, 147–171 (2006)

    Article  Google Scholar 

  26. Jacovina, Matthew E., Tanner Jackson, G., Snow, Erica L., McNamara, Danielle S.: Timing game-based practice in a reading comprehension strategy tutor. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 59–68. Springer, Cham (2016). doi:10.1007/978-3-319-39583-8_6

    Google Scholar 

  27. Jackson, G.T., Boonthum, C., McNamara, D.S.: iSTART-ME: situating extended learning within a game-based environment. In: Lane, H.C., Ogan, A., Shute, V. (eds.) Proceedings of the Workshop on Intelligent Educational Games at the 14th Annual Conference on Artificial Intelligence in Education, AIED, Brighton, UK, pp. 59–68 (2009)

    Google Scholar 

  28. McNamara, D.S., Jacovina, M.E., Snow, E.L., Allen, L.K.: From generating in the lab to tutoring systems in classrooms. Am. J. Psychol. 128, 159–172 (2015)

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported in part by the Institute of Education Sciences (R305A130124). Any opinions, conclusions, or recommendations expressed are those of the authors and do not necessarily represent views of the IES.

Author information

Authors and Affiliations

  1. Institute for the Science of Teaching and Learning, Arizona State University, Tempe, AZ, USA

    Kathryn S. McCarthy, Matthew E. Jacovina, Tricia A. Guerrero & Danielle S. McNamara

  2. Center for Technology in Learning, SRI International, Menlo Park, CA, USA

    Erica L. Snow

Authors
  1. Kathryn S. McCarthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew E. Jacovina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erica L. Snow
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tricia A. Guerrero
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Danielle S. McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathryn S. McCarthy .

Editor information

Editors and Affiliations

  1. Human-Centered Multimedia, University of Augsburg, Augsburg, Germany

    Elisabeth André

  2. University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Ryan Baker

  3. Department of Psychology, University of Memphis, Memphis, Tennessee, USA

    Xiangen Hu

  4. Ateneo de Manila University, Quezon City, Philippines

    Ma. Mercedes T. Rodrigo

  5. University of Sussex, Brighton, United Kingdom

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

McCarthy, K.S., Jacovina, M.E., Snow, E.L., Guerrero, T.A., McNamara, D.S. (2017). iSTART Therefore I Understand: But Metacognitive Supports Did not Enhance Comprehension Gains. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61425-0_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61424-3

  • Online ISBN: 978-3-319-61425-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation