Skip to main content
SpringerLink
Log in

Dynamics of learning: time-varying feedback effects within the intelligent tutoring system of structure strategy (ITSS)

  • Research Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

The intelligent tutoring system of structure strategy (ITSS) is a web-based digital tutoring system proven to be effective in helping students recognize and use text structures to comprehend and recall texts. However, little is known about the dynamic learning processes within the ITSS. This study aims to investigate the effects of feedback dosage on lesson mastery throughout the progression of ITSS lessons. We applied a confirmatory factor analysis and extended vector autoregressive model to assess the dynamic relationships among three tasks embedded within the ITSS and found: (1) significant cross-regression effects among the three reading tasks; (2) distinct effects of feedback dosage on the specific reading task; and (3) different effect sizes of feedback across lessons. Results provide helpful insights on ways to design better modules in further development of the ITSS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The participants were allowed to make up to 6 attempts in the ITSS system. Only data from the first 4 attempts were included in the analysis because most participants exceeded the minimum threshold (scoring at least 80 out of 100) to proceed to the next item by the 4th attempt, thus generating excessive missingness if we were to include data from all 6 attempts.

References

  • Baddeley, A., Papagno, C., & Vallar, G. (1988). When long-term learning depends on short-term storage. Journal of Memory and Language, 27(5), 586–595.

    Article  Google Scholar 

  • Bar-Shalom, Y., Li, X. R., & Kirubarajan, T. (2001). Estimation with applications to tracking and navigation: {T}heory algorithms and software. Wiley.

    Book  Google Scholar 

  • Bringmann, L. F., Ferrer, E., Hamaker, E. L., Borsboom, D., & Tuerlinckx, F. (2018). Modeling nonstationary emotion dynamics in dyads using a time-varying vector-autoregressive model. Multivariate Behavioral Research, 3171, 1–22.

    Google Scholar 

  • Chen, M., Chow, S.-M., Hammal, Z., Messinger, D. S., & Cohn, J. F. (2020). A person-and time-varying vector autoregressive model to capture interactive infant-mother head movement dynamics. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2020.1762065

    Article  Google Scholar 

  • Chen, M., Chow, S.-M., & Hunter, M. D. (2018). Stochastic differential equation models with time-varying parameters. Continuous time modeling in the behavioral and related sciences (pp. 205–238). Springer.

    Chapter  Google Scholar 

  • Chow, S. M., Grimm, K. J., Filteau, G., Dolan, C. V., & McArdle, J. J. (2013). Regime-switching bivariate dual change score model. Multivariate Behavioral Research, 48(4), 463–502. https://doi.org/10.1080/00273171.2013.787870

    Article  Google Scholar 

  • Chow, S.-M., Haltigan, J. D., & Messinger, D. S. (2010). Dynamic patterns of infant-parent interactions during face-to-face and still-face episodes. Emotion, 10(1), 101–114.

    Article  Google Scholar 

  • Chow, S.-M., Hamagami, F., & Nesselroade, J. R. (2007). Age differences in dynamical cognition-emotion linkages. Psychology and Aging, 22(4), 765–780.

    Article  Google Scholar 

  • Chow, S. M., Zu, J., Shifren, K., & Zhang, G. (2011). Dynamic factor analysis models with time-varying parameters. Multivariate Behavioral Research, 46(2), 303–339. https://doi.org/10.1080/00273171.2011.563697

    Article  Google Scholar 

  • Clarke, B., Baker, S., Smolkowski, K., & Chard, D. J. (2008). An analysis of early numeracy curriculum-based measurement: Examining the role of growth in student outcomes. Remedial and Special Education, 29(1), 46–57. https://doi.org/10.1177/0741932507309694

    Article  Google Scholar 

  • Dalton, B., & Proctor, C. P. (2007). Reading as thinking: Integrating strategy instruction in a universally designed digital literacy environment. In D. S. McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and technologies (pp. 421–440). Erlbaum.

    Google Scholar 

  • De Bot, K. (2008). Introduction: Second language development as a dynamic process. The Modern Language Journal, 92, 166–178. https://doi.org/10.1111/j.1540-4781.2008.00712.x

    Article  Google Scholar 

  • Du Toit, S. H. C., & Browne, M. W. (2007). Structural equation modeling of multivariate time series. Multivariate Behavioral Research, 42, 67–101.

    Article  Google Scholar 

  • Fan, J., & Zhang, W. (2008). Statistical methods with varying coefficient models. Statistics and Its Interface, 1, 2.

    Google Scholar 

  • Goegebeur, Y., De Boeck, P., Wollack, J. A., & Cohen, A. S. (2008). A speeded item response model with gradual process change. Psychometrika, 73(1), 65–87. https://doi.org/10.1007/s11336-007-9031-2

    Article  Google Scholar 

  • Hamilton, J. D. (1994). Time series analysis. Princeton University Press.

    Book  Google Scholar 

  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research., 77, 81–112.

    Article  Google Scholar 

  • Horn, J. L., & Little, K. B. (1966). Isolating change and invariance in patterns of behavior. Multivariate Behavioral Research, 1, 219–222.

    Article  Google Scholar 

  • Hung, L. F., & Wang, W. C. (2012). The generalized multilevel facets model for longitudinal data. Journal of Educational and Behavioral Statistics, 37(2), 231–255. https://doi.org/10.3102/1076998611402503

    Article  Google Scholar 

  • Ji, L., & Chow, S.-M. (2018). Methodological issues and extensions to the latent difference score framework 1. In E. Ferrer, S. M. Boker, & K. J. Grimm (Eds.), Longitudinal multivariate psychology (pp. 9–37). Routledge.

    Google Scholar 

  • Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343

    Article  Google Scholar 

  • Joshi, R. M., & Aaron, P. G. (2000). The component model of reading: Simple view of reading made a little more complex. Reading Psychology, 21(2), 85–97. https://doi.org/10.1080/02702710050084428

    Article  Google Scholar 

  • Kim, S., & Camilli, G. (2014). An item response theory approach to longitudinal analysis with application to summer setback in preschool language/literacy. Large-Scale Assessments in Education, 2(1), 1–17. https://doi.org/10.1186/2196-0739-2-1

    Article  Google Scholar 

  • Krause, U., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19(2), 158–170.

    Article  Google Scholar 

  • Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. https://doi.org/10.3102/0034654315581420

    Article  Google Scholar 

  • Lemke, J. L. (2002). Language development and identity: Multiple timescales in the social ecology of learning (pp. 68–87). Language Acquisition and Language Socialization.

    Google Scholar 

  • Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Welsley Publishing Company.

    Google Scholar 

  • Lütkepohl, H. (2005). Introduction to multiple time series analysis (2nd ed.). Springer-Verlag.

    Book  Google Scholar 

  • Lysakowski, R. S., & Walberg, H. J. (1982). Instructional effects of cues, participation, and corrective feedback: A quantitative synthesis. American Educational Research Journal, 19(4), 559–572. https://doi.org/10.3102/00028312019004559

    Article  Google Scholar 

  • Mason, L. H. (2013). Teaching students who struggle with learning to think before, while, and after reading: Effects of SRSD instruction. Reading and Writing Quarterly, 29, 124–144.

    Article  Google Scholar 

  • Maulana, R., Opdenakker, M. C., Stroet, K., & Bosker, R. (2013). Changes in teachers’ involvement versus rejection and links with academic motivation during the first year of secondary education: A multilevel growth curve analysis. Journal of Youth and Adolescence, 42(9), 1348–1371. https://doi.org/10.1007/s10964-013-9921-9

    Article  Google Scholar 

  • Meyer, B. J. F. (1975). The organization of prose and its effects on memory. North—Holland Press.

    Google Scholar 

  • Meyer, B. J. F., & Poon, L. W. (2001). Effects of the structure strategy and signaling on recall of text. Journal of Educational Psychology, 93, 141–159.

    Article  Google Scholar 

  • Meyer, B. J. F., Wijekumar, K., & Lei, P. (2018). Comparative signaling generated for expository texts by 4th–8th graders: Variations by text structure strategy instruction, comprehension skill, and signal word. Reading and Writing, 31(9), 1937–1968. https://doi.org/10.1007/s11145-018-9871-4

    Article  Google Scholar 

  • Meyer, B. J. F., Wijekumar, K., Middlemiss, W., Higley, K., Lei, P.-W., Meier, C., & Spielvogel, J. (2010). Web-based tutoring of the structure strategy with or without elaborated feedback or choice for fifth-and seventh-grade readers. Reading Research Quarterly, 45(1), 62–92.

    Article  Google Scholar 

  • Meyer, B. J. F., Young, C. J., & Bartlett, B. J. (1989). Memory improved: Reading and memory enhancement across the life span through strategic text structures. Erlbaum.

    Google Scholar 

  • Molenaar, P. C. M. (1994). Dynamic factor analysis of psychophysiological signals. In J. R. Jennings, P. K. Ackles, & M. G. H. Coles (Eds.), Advances in psychophysiology: A research annual (Vol. 5, pp. 229–302). Jessica Kingsley Publishers.

    Google Scholar 

  • Molenaar, P. C. M., De Gooijer, J. G., & Schmitz, B. (1992). Dynamic factor analysis of nonstationary multivariate time series. Psychometrika, 57(3), 333–349. https://doi.org/10.1007/BF02295422

    Article  Google Scholar 

  • Molenaar, P., Sinclair, K. O., Rovine, M. J., Ram, N., & Corneal, S. E. (2009). Analyzing developmental processes on an individual level using nonstationary time series modeling. Developmental Psychology, 45(1), 260.

    Article  Google Scholar 

  • Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32, 99–113.

    Article  Google Scholar 

  • Murphy, P. (2007). Reading comprehension exercises online: The effects of feedback, proficiency and interaction. Language Learning & Technology, 11(3), 107–129.

    Google Scholar 

  • National Assessment of Educational Progress (NAEP). (2007). Retrieved December 20, 2008, from https://www.nationsreportcard.gov/reading_2007/r0003.asp

  • Nicol, D. (2010). From monologue to dialogue: Improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education., 35, 501–517. https://doi.org/10.1080/02602931003786559

    Article  Google Scholar 

  • Ou, L., Chow, S.-M., Ji, L., & Molenaar, P. C. M. (2017). (Re)evaluating the implications of the autoregressive latent trajectory model through likelihood ratio tests of its initial conditions. Multivariate Behavioral Research, 52, 178–199.

    Article  Google Scholar 

  • Ou, L., Hunter, M. D., & Chow, S. M. (2019). What’s for dynr: A package for linear and nonlinear dynamic modeling in R. The R Journal, 11(1), 91–111.

    Article  Google Scholar 

  • Pagan, A. (1980). Some identification and estimation results for regression models with stochastically varying coefficients. Journal of Econometrics, 13, 341–363. https://doi.org/10.1016/0304-4076(80)90084-6

    Article  Google Scholar 

  • Perfetti, C., & Stafura, J. (2014). Word knowledge in a theory of reading comprehension. Scientific Studies of Reading, 18, 22–37. https://doi.org/10.1080/10888438.2013.827687

    Article  Google Scholar 

  • Raphael, T. E., & Kirschner, B. M. (1985). The effects of instruction in compare/contrast text structure on sixth-grade students' reading comprehension and writing products. Research Series No. 161.

  • Rijmen, F., De Boeck, P., & Van Der Maas, H. L. J. (2005). An IRT model with a parameter-driven process for change. Psychometrika, 70(4), 651–669. https://doi.org/10.1007/s11336-002-1047-z

    Article  Google Scholar 

  • Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

    Article  Google Scholar 

  • Shevlin, M., & Millar, R. (2006). Career education: An application of latent growth curve modelling to career information-seeking behaviour of school pupils. British Journal of Educational Psychology, 76(1), 141–153. https://doi.org/10.1348/000709904x22386

    Article  Google Scholar 

  • Stoel, R. D., Peetsma, T. T. D., & Roeleveld, J. (2003). Relations between the development of school investment, self-confidence, and language achievement in elementary education: A multivariate latent growth curve approach. Learning and Individual Differences, 13(4), 313–333. https://doi.org/10.1016/S1041-6080(03)00017-7

    Article  Google Scholar 

  • Tenenbaum, G., & Goldring, E. (1989). A meta-analysis of the effect of enhanced instruction: Cues, participation, reinforcement and feedback and correctives on motor skill learning. Journal of Research and Development in Education, 22, 53–64.

    Google Scholar 

  • Van de Gaer, E., De Fraine, B., Pustjens, H., Van Damme, J., De Munter, A., & Onghena, P. (2009). School effects on the development of motivation toward learning tasks and the development of academic self-concept in secondary education: A multivariate latent growth curve approach. School Effectiveness and School Improvement, 20(2), 235–253.

    Article  Google Scholar 

  • Van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. Academic Press.

    Google Scholar 

  • Vollmeyer, R., & Rheinberg, F. (2005). A surprising effect of feedback on learning. Learning and Instruction, 15, 589–602. https://doi.org/10.1016/j.learninstruc.2005.08.001

    Article  Google Scholar 

  • Wang, Q., Molenaar, P., Harsh, S., Freeman, K., Xie, J., Gold, C., Rovine, M., & Ulbrecht, J. (2014). Personalized state-space modeling of glucose dynamics for type 1 diabetes using continuously monitored glucose, insulin dose, and meal intake: An extended Kalman filter approach. Journal of Diabetes Science and Technology, 8(2), 331–345. https://doi.org/10.1177/1932296814524080

    Article  Google Scholar 

  • Waninge, F., Dörnyei, Z., & De Bot, K. (2014). Motivational dynamics in language learning: Change, stability, and context. The Modern Language Journal. https://doi.org/10.1111/j.1540-4781.2014.12118.x

    Article  Google Scholar 

  • Wiederholt, J. L., & Blalock, G. (2000). Gray silent reading tests. Pro-Ed.

    Google Scholar 

  • Wijekumar, K., Meyer, B. J. F., & Lei, P. (2012). Large-scale randomized controlled trial with 4th graders using intelligent tutoring of the structure strategy to improve nonfiction reading comprehension. Journal of Educational Technology Research and Development., 60, 987–1013.

    Article  Google Scholar 

  • Wijekumar, K., Meyer, B. J. F., & Lei, P. (2017a). Web-based text structure strategy instruction improves seventh graders’ content area reading comprehension. Journal of Educational Psychology, 109(6), 741–760.

    Article  Google Scholar 

  • Wijekumar, K., Meyer, B. J. F., Lei, P., Cheng, W., Ji, X., & Joshi, R. M. (2017b). Evidence of an intelligent tutoring system as a mindtool to promote strategic memory of expository texts and comprehension with children in grades 4 and 5. Journal of Educational Computing Research, 55(7), 1022–1048. https://doi.org/10.1177/0735633117696909

    Article  Google Scholar 

  • Wijekumar, K., Meyer, B. J. F., Lei, P., Lin, Y., Johnson, L. A., Spielvogel, J. A., et al. (2014). Multisite randomized controlled trial examining intelligent tutoring of structure strategy for fifth-grade. Journal of Research on Educational Effectiveness, 7, 331–357. https://doi.org/10.1080/19345747.2013.853333

    Article  Google Scholar 

  • Wijekumar, K., Zhang, S., Joshi, R. M., & PetiStantic, A. (2021). Introduction to the special issue: Textbook content and organization—why it matters to reading comprehension in elementary grades? Technology, Knowledge and Learning, 26(2), 243–249.

    Article  Google Scholar 

  • Williams, J. P., Hall, K. M., Lauer, K. D., DeSisto, L. A., deCani, J. S., & Stafford, K. B. (2005). Expository text comprehension in the primary grade classroom. Journal of Educational Psychology, 97(4), 538–550. https://doi.org/10.1037/0022-0663.97.4.538

    Article  Google Scholar 

  • Woolf, B. P. (2009). Building intelligent interactive tutors: Student centered strategies for revolutionizing e-learning. Elsevier.

    Google Scholar 

Download references

Acknowledgements

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education [grant number R305A080133] and National Science Foundation [grant number IGE-1806874]. The opinions expressed are those of the authors and do not represent views of the Institutes or the U.S. Department of Education.

Funding

This study was funded by the Institute of Education Sciences, U.S. Department of Education (grant number R305A080133) and and National Science Foundation [grant number IGE-1806874].

Author information

Authors and Affiliations

  1. College of Health and Human Development, The Pennsylvania State University, State College, PA, USA

    Jungmin Lee, Sy-Miin Chow & Peter C. M. Molenaar

  2. College of Education, The Pennsylvania State University, State College, PA, USA

    Puiwa Lei

  3. College of Education and Human Development, Texas A&M University, College Station, TX, USA

    Kausalai (Kay) Wijekumar

Authors
  1. Jungmin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sy-Miin Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Puiwa Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kausalai (Kay) Wijekumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter C. M. Molenaar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jungmin Lee.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee (Penn State IRB #28763). This was a secondary data analysis with de-identified data from computer log files.

Informed consent

Informed consent was obtained from all individual participants included in the study. Consent was obtained for the original study under the IRB approval 28763.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, J., Chow, SM., Lei, P. et al. Dynamics of learning: time-varying feedback effects within the intelligent tutoring system of structure strategy (ITSS). Education Tech Research Dev 69, 2963–2984 (2021). https://doi.org/10.1007/s11423-021-10049-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-021-10049-w

Keywords

Search

Navigation