Skip to main content
SpringerLink
Log in

Improving Reading Comprehension with Automatically Generated Cloze Item Practice

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10331))

Included in the following conference series:

Abstract

This study investigated the effect of cloze item practice on reading comprehension, where cloze items were either created by humans, by machine using natural language processing techniques, or randomly. Participants from Amazon Mechanical Turk (\(N=302\)) took a pre-test, read a text, and took part in one of five conditions, Do-Nothing, Re-Read, Human Cloze, Machine Cloze, or Random Cloze, followed by a 24-hour retention interval and post-test. Participants used the MoFaCTS system [27], which in cloze conditions presented items adaptively based on individual success with each item. Analysis revealed that only Machine Cloze was significantly higher than the Do-Nothing condition on post-test, \(d=.58\), \(CI_{95} [.21,.94]\). Additionally, Machine Cloze was significantly higher than Human and Random Cloze conditions on post-test, \(d=.49\), \(CI_{95} [.12,.86]\) and \(d=.71\), \(CI_{95} [.34,1.09]\) respectively. These results suggest that Machine Cloze items generated using natural language processing techniques are effective for enhancing reading comprehension when delivered by an adaptive practice scheduling system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmed, Y., Francis, D.J., York, M., Fletcher, J.M., Barnes, M., Kulesz, P.: Validation of the direct and inferential mediation (DIME) model of reading comprehension in grades 7 through 12. Contemp. Educ. Psychol. 4445, 68–82 (2016)

    Article  Google Scholar 

  2. Aleven, V., Mclaren, B.M., Sewall, J., Koedinger, K.R.: A new paradigm for intelligent tutoring systems: example-tracing tutors. Int. J. Artif. Intell. Educ. 19(2), 105–154 (2009)

    Google Scholar 

  3. Bransford, J.D., Johnson, M.K.: Contextual prerequisites for understanding: some investigations of comprehension and recall. J. Verbal Learn. Verbal Behav. 11(6), 717–726 (1972)

    Article  Google Scholar 

  4. Britton, B.K., Gülgöz, S.: Using Kintsch’s computational model to improve instructional text: effects of repairing inference calls on recall and cognitive structures. J. Educ. Psychol. 83(3), 329–345 (1991)

    Article  Google Scholar 

  5. Chall, J.S., Jacobs, V.A.: Writing and reading in the elementary grades: developmental trends among low SES children. Lang. Arts 60(5), 617–626 (1983). http://www.jstor.org/stable/41961511

    Google Scholar 

  6. Chi, M.T.H., Wylie, R.: The ICAP framework: linking cognitive engagement to active learning outcomes. Educ. Psychol. 49(4), 219–243 (2014). http://dx.doi.org/10.1080/00461520.2014.965823

    Article  Google Scholar 

  7. Fang, Z.: The language demands of science reading in middle school. Int. J. Sci. Educ. 28(5), 491–520 (2006). http://dx.doi.org/10.1080/09500690500339092

    Article  Google Scholar 

  8. Harrell, F.: Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. Graduate Texts in Mathematics. Springer, New York (2001)

    Book  MATH  Google Scholar 

  9. Hirsch, E.D.: Reading comprehension requires knowledge-of words and the world. Am. Educator 27(1), 10–13, 16–22, 28–29, 48 (2003)

    Google Scholar 

  10. Johansson, R., Nugues, P.: Dependency-based syntactic-semantic analysis with PropBank and NomBank. In: Proceedings of the Twelfth Conference on Computational Natural Language Learning, CoNLL 2008, pp. 183–187. Association for Computational Linguistics, Morristown (2008)

    Google Scholar 

  11. Kelly, S., Ye, F.: Accounting for the relationship between initial status and growth in regression models. J. Exp. Educ. 85(3), 353–375 (2017). http://dx.doi.org/10.1080/00220973.2016.1160357

    Article  Google Scholar 

  12. Kintsch, W.: Comprehension: A Paradigm for Cognition. Cambridge University Press, Cambridge (1998)

    Google Scholar 

  13. Laufer, B.: Lexical thresholds for reading comprehension: what they are and how they can be used for teaching purposes. TESOL Q. 47(4), 867–872 (2013). http://www.jstor.org/stable/43267941

    Article  Google Scholar 

  14. Leelawong, K., Biswas, G.: Designing learning by teaching agents: the Betty’s Brain system. Int. J. Artif. Intell. Educ. 18(3), 181–208 (2008)

    Google Scholar 

  15. Lipson, M.Y.: Learning new information from text: the role of prior knowledge and reading ability. J. Read. Behav. 14(3), 243–261 (1982)

    Article  Google Scholar 

  16. Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60. Association for Computational Linguistics, Baltimore, June 2014. http://www.aclweb.org/anthology/pp. 14-5010

  17. McKeown, M.G., Beck, I.L., Omanson, R.C., Pople, M.T.: Some effects of the nature and frequency of vocabulary instruction on the knowledge and use of words. Read. Res. Q. 20(5), 522–535 (1985). http://www.jstor.org/stable/747940

    Article  Google Scholar 

  18. McNamara, D., Levinstein, I., Boonthum, C.: iSTART: interactive strategy training for active reading and thinking. Behav. Res. Methods 36, 222–233 (2004). doi:10.3758/BF03195567

    Article  Google Scholar 

  19. McNamara, D.S., Magliano, J.: Toward a comprehensive model of comprehension. In: The Psychology of Learning and Motivation, Psychology of Learning and Motivation, vol. 51, pp. 297–384. Academic Press (2009). https://www.sciencedirect.com/science/article/pii/S0079742109510092

  20. Mol, S.E., Bus, A.G.: To read or not to read: a meta-analysis of print exposure from infancy to early adulthood. Psychol. Bull. 137(2), 267–296 (2011)

    Article  Google Scholar 

  21. Moss, B.: Making a case and a place for effective content area literacy instruction in the elementary grades. Read. Teach. 59(1), 46–55 (2005). http://dx.doi.org/10.1598/RT.59.1.5

    Article  Google Scholar 

  22. Nagy, W., Townsend, D.: Words as tools: learning academic vocabulary as language acquisition. Read. Res. Q. 47(1), 91–108 (2012). http://dx.doi.org/10.1002/RRQ.011

    Article  Google Scholar 

  23. National Institute of Child Health and Human Development: Report of the National Reading Panel. Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. NIH Publication No. 00-4769, U.S. Government Printing Office, Washington, DC (2000)

    Google Scholar 

  24. Nye, B.D., Graesser, A.C., Hu, X.: AutoTutor and family: a review of 17 years of natural language tutoring. Int. J. Artif. Intell. Educ. 24(4), 427–469 (2014). http://dx.doi.org/10.1007/s40593-014-0029-5

    Article  Google Scholar 

  25. Olney, A.M., Brawner, K., Pavlik, P., Koedinger, K.: Emerging trends in automated authoring. In: Sottilare, R., Graesser, A., Hu, X., Brawner, K. (eds.) Design Recommendations for Intelligent Tutoring Systems, Adaptive Tutoring, vol. 3, pp. 227–242. U.S. Army Research Laboratory, Orlando (2015)

    Google Scholar 

  26. Ozuru, Y., Dempsey, K., McNamara, D.S.: Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learn. Instr. 19(3), 228–242 (2009)

    Article  Google Scholar 

  27. Pavlik, P.I., Kelly, C., Maass, J.K.: The mobile fact and concept training system (MoFaCTS). In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 247–253. Springer, Cham (2016). doi:10.1007/978-3-319-39583-8_25

    Google Scholar 

  28. Recht, D.R., Leslie, L.: Effect of prior knowledge on good and poor readers’ memory of text. J. Educ. Psychol. 80(1), 16–20 (1988)

    Article  Google Scholar 

  29. Surdeanu, M., Hicks, T., Valenzuela-Escarcega, M.A.: Two practical rhetorical structure theory parsers. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pp. 1–5. Association for Computational Linguistics, Denver. http://www.aclweb.org/anthology/N15-3001

  30. Torgesen, J.K.: Avoiding the devastating downward spiral: the evidence that early intervention prevents reading failure. Am. Educator 28(3), 6–19 (2004)

    Google Scholar 

  31. Vidal-Abarca, E., Martínez, G., Gilabert, R.: Two procedures to improve instructional text: effects on memory and learning. J. Educ. Psychol. 92(1), 107–116 (2000)

    Article  Google Scholar 

  32. What Works Clearinghouse: Assessing attrition bias. Technical report, Institute of Eduation Sciences (2012). https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_attrition_v2.1.pdf

  33. Wolfe, M.B., Schreiner, M., Rehder, B., Laham, D., Foltz, P.W., Kintsch, W., Landauer, T.K.: Learning from text: matching readers and texts by latent semantic analysis. Discourse Process. 25(2–3), 309–336 (1998). http://dx.doi.org/10.1080/01638539809545030

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science Foundation Data Infrastructure Building Blocks program (NSF; ACI-1443068), by the Institute of Education Sciences (IES; R305C120001) and by the Office of Naval Research (ONR; N00014-00-1-0600, N00014-12-C-0643; N00014-16-C-3027). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF, IES, or ONR.

Author information

Authors and Affiliations

  1. University of Memphis, Memphis, TN, 38152, USA

    Andrew M. Olney, Philip I. Pavlik Jr. & Jaclyn K. Maass

Authors
  1. Andrew M. Olney
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Philip I. Pavlik Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jaclyn K. Maass
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew M. Olney .

Editor information

Editors and Affiliations

  1. Human-Centered Multimedia, University of Augsburg, Augsburg, Germany

    Elisabeth André

  2. University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Ryan Baker

  3. Department of Psychology, University of Memphis, Memphis, Tennessee, USA

    Xiangen Hu

  4. Ateneo de Manila University, Quezon City, Philippines

    Ma. Mercedes T. Rodrigo

  5. University of Sussex, Brighton, United Kingdom

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Olney, A.M., Pavlik, P.I., Maass, J.K. (2017). Improving Reading Comprehension with Automatically Generated Cloze Item Practice. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61425-0_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61424-3

  • Online ISBN: 978-3-319-61425-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation