Skip to main content
SpringerLink
Log in

Assessing Question Quality Using NLP

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10331))

Included in the following conference series:

Abstract

An NLP algorithm was developed to assess question quality to inform feedback on questions generated by students within iSTART (an intelligent tutoring system that teaches reading strategies). A corpus of 4575 questions was coded using a four-level taxonomy. NLP indices were calculated for each question and machine learning was used to predict question quality. NLP indices related to lexical sophistication modestly predicted question type. Accuracies improved when predicting two levels (shallow versus deep).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. McNamara, D.S.: Self-explanation and reading strategy training (SERT) improves low-knowledge students’ science course performance. Discourse Process. 38, 1–30 (2015)

    Article  Google Scholar 

  2. McNamara, D.S., O’Reilly, T.P., Best, R.M., Ozuru, Y.: Improving adolescent learners’ reading comprehension with iSTART. J. Educ. Comput. Res. 34(2), 147–171 (2006)

    Article  Google Scholar 

  3. Cerdán, R., Vidal-Abarca, E., Martínez, T., Gilabert, R., Gil, L.: Impact of question-answering tasks on search processes and reading comprehension. Learn. Instr. 19(1), 13–27 (2009)

    Article  Google Scholar 

  4. Graesser, A. C., Jackson, G.T., Mathews, E.C., Mitchell, H.H., Olney, A., Ventura, M., Chipman, P., Franceschetti, D., Hu, X., Louwerse, M.M., Person, N.K.: Why/autotutor: a test of learning gains from a physics tutor with natural language dialog. In: Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society, pp. 1–6 (2003)

    Google Scholar 

  5. Graesser, A.C., Person, N.K.: Question asking during tutoring. Am. Educ. Res. J. 31(1), 104–137 (1994)

    Article  Google Scholar 

  6. McNamara, D.S., Allen, L.K., Crossley, S.A., Dascalu, M., Perret, C.A.: Natural language processing and learning analytics. In: Siemens, G., Lang, C. (eds.) Handbook of Learning Analytics and Educational Data Mining (in press)

    Google Scholar 

  7. Crossley, S., Kyle, K., Davenport, J., McNamara, D.S.: Automatic assessment of constructed response data in a chemistry tutor. In: Barnes, T., Chi, M., Feng, M. (eds.) EDM 2016, pp. 336–340. International Educational Data Mining Society, Raleigh (2016)

    Google Scholar 

Download references

Acknowledgments

This research was supported in part by the Institute for Educational Sciences (IES R305A130124) and the Office of Naval Research (ONR N00014-14-1-0343 and ONR N00014-17-1-2300). Any opinions, findings, and conclusions or recommendations expressed are those of the authors and do not necessarily reflect the views of IES or ONR.

Author information

Authors and Affiliations

  1. Department of Psychology, Arizona State University, Tempe, AZ, 85287, USA

    Kristopher J. Kopp, Amy M. Johnson & Danielle S. McNamara

  2. Department of Applied Linguistics/ESL, Georgia State University, 25 Park Place, Atlanta, GA, 30303, USA

    Scott A. Crossley

Authors
  1. Kristopher J. Kopp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amy M. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Scott A. Crossley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danielle S. McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristopher J. Kopp .

Editor information

Editors and Affiliations

  1. Human-Centered Multimedia, University of Augsburg, Augsburg, Germany

    Elisabeth André

  2. University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Ryan Baker

  3. Department of Psychology, University of Memphis, Memphis, Tennessee, USA

    Xiangen Hu

  4. Ateneo de Manila University, Quezon City, Philippines

    Ma. Mercedes T. Rodrigo

  5. University of Sussex, Brighton, United Kingdom

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Kopp, K.J., Johnson, A.M., Crossley, S.A., McNamara, D.S. (2017). Assessing Question Quality Using NLP. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_55

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61425-0_55

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61424-3

  • Online ISBN: 978-3-319-61425-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation