Abstract
Computerized continuous performance tests (CPTs) are commonly used to characterize attention in attention deficit-hyperactivity disorder (ADHD). Virtual classroom CPTs, designed to enhance ecological validity, are increasingly being utilized. Lacking is a quantitative meta-analysis of clinical comparisons of attention performance in children with ADHD using virtual classroom CPTs. The objective of the present systematic PRISMA review was to address this empirical void and compare three-dimensional (3D) virtual classroom CPTs to traditional two-dimensional (2D) CPTs. The peer-reviewed literature on comparisons of virtual classroom performance between children with ADHD and typically developing children was explored in six databases (e.g., Medline). Published studies using a virtual classroom to compare attentional performance between children with ADHD and typically developing children were included. Given the high heterogeneity with modality comparisons (i.e., computerized CPTs vs. virtual classroom CPTs for ADHD), both main comparisons included only population comparisons (i.e., control vs. ADHD) using each CPT modality. Meta-analytic findings were generally consistent with previous meta-analyses of computerized CPTs regarding the commonly used omission, commission, and hit reaction time variables. Results suggest that the virtual classroom CPTs reliably differentiate attention performance in persons with ADHD. Ecological validity implications are discussed pertaining to subtle meta-analytic outcome differences compared to computerized 2D CPTs. Further, due to an inability to conduct moderator analyses, it remains unclear if modality differences are due to other factors. Suggestions for future research using the virtual classroom CPTs are provided.
Similar content being viewed by others
References
Adams, R., Finn, P., Moes, E., Flannery, K., & Rizzo, A. S. (2009). Distractibility in attention/deficit/hyperactivity disorder (ADHD): The virtual reality classroom. Child Neuropsychology, 15(2), 120–135.
Areces, D., Rodríguez, C., García, T., Cueli, M., & González-Castro, P. (2016). Efficacy of a continuous performance test based on virtual reality in the diagnosis of ADHD and its clinical presentations. Journal of Attention Disorders, 1–11. https://doi.org/10.1177/1087054716629711
Areces, D., Rodríguez, C., García, T., Cueli, M., & González-Castro, P. (2018a). Efficacy of a continuous performance test based on virtual reality in the diagnosis of ADHD and its clinical presentations. Journal of Attention Disorders, 22(11), 1081–1091.
Ballard, J. C. (2001). Assessing attention: Comparison of response-inhibition and traditional continuous performance tests. Journal of Clinical and Experimental Neuropsychology, 23(3), 331–350.
Barkley, R. A. (1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19(2), 149–178.
Barkley, R. A., & Murphy, K. R. (2011). The nature of executive function (EF) deficits in daily life activities in adults with ADHD and their relationship to performance on EF tests. Journal of Psychopathology and Behavioral Assessment, 33(2), 137–158.
Beale, I. L., Matthew, P. J., Oliver, S., & Corballis, M. C. (1987). Performance of disabled and normal readers on the continuous performance test. Journal of Abnormal Child Psychology, 15(2), 229–238.
Berger, I., Slobodin, O., & Cassuto, H. (2017). Usefulness and validity of continuous performance tests in the diagnosis of attention-deficit hyperactivity disorder children. Archives of Clinical Neuropsychology, 32(1), 81–93.
Bioulac, S., Lallemand, S., Rizzo, A., Philip, P., Fabrigoule, C., & Bouvard, M. P. (2012). Impact of time on task on ADHD patient’s performances in a virtual classroom. European Journal of Paediatric Neurology, 16(5), 514–521.
Bioulac, S., Micoulaud-Franchi, J. A., Maire, J., Bouvard, M. P., Rizzo, A. A., Sagaspe, P., & Philip, P. (2018). Virtual remediation versus methylphenidate to improve distractibility in children with ADHD: A controlled randomized clinical trial study. Journal of Attention Disorders, 1087054718759751.
Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12(12), 752.
Borenstein, M., Higgins, J. P., Hedges, L. V., & Rothstein, H. R. (2017). Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Research Synthesis Methods, 8(1), 5–18.
Brosco, J. P., & Bona, A. (2016). Changes in academic demands and attention-deficit/hyperactivity disorder in young children. JAMA Pediatrics, 170(4), 396–397.
Bruce, K. S. (2011). Academic underachievement: A neurodevelopmental perspective. Revista Médica Clínica Las Condes, 22(2), 211–217.
Chaytor, N., Schmitter-Edgecombe, M., & Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21(3), 217–227.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.
Cohn, L. D., & Becker, B. J. (2003). How meta-analysis increases statistical power. Psychological Methods, 8, 243–253.
Conners, C. K., Epstein, J. N., Angold, A., & Klaric, J. (2003). Continuous performance test performance in a normative epidemiological sample. Journal of Abnormal Child Psychology, 31(5), 555–562.
Corkum, P. V., & Siegel, L. S. (1993). Is the continuous performance task a valuable research tool for use with children with attention-deficit-hyperactivity disorder? Journal of Child Psychology and Psychiatry, 34(7), 1217–1239.
Díaz-Orueta, U. (2017). Advances in neuropsychological assessment of attention. In R. Kane & T. Parsons (Eds.), The role of Technology in Clinical Neuropsychology (pp. 103–142). Oxford: Oxford University Press.
Díaz-Orueta, U., Garcia-López, C., Crespo-Eguílaz, N., Sánchez-Carpintero, R., Climent, G., & Narbona, J. (2014). AULA virtual reality test as an attention measure: Convergent validity with Conners’ continuous performance test. Child Neuropsychology, 20(3), 328–342.
Duffield, T., Parsons, T. D., Karam, S., Otero, T., & Hall, T. (2018). Virtual reality as an assessment modality with pediatric ASD populations: A systematic review. Child Neuropsychology.
Fasmer, O. B., Mjeldheim, K., Førland, W., Hansen, A. L., Syrstad, V. E. G., Oedegaard, K. J., & Berle, J. Ø. (2016). Linear and non-linear analyses of Conner’s continuous performance test-II discriminate adult patients with attention deficit hyperactivity disorder from patients with mood and anxiety disorders. BMC Psychiatry, 16(1), 284.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.
Franzen, M. D., & Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychology. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 91–112). Delray Beach, FL: GR Press/St. Lucie Press.
Gilboa, Y., Kerrouche, B., Longaud-Vales, A., Kieffer, V., Tiberghien, A., Aligon, D., … Paule Chevignard, M. (2015). Describing the attention profile of children and adolescents with acquired brain injury using the virtual classroom. Brain Injury, 29(13–14), 1691–1700.
Gilboa, Y., Rosenblum, S., Fattal-Valevski, A., Toledano-Alhadef, H., & Josman, N. (2011). Using a virtual classroom environment to describe the attention deficits profile of children with neurofibromatosis type 1. Research in Developmental Disabilities, 32(6), 2608–2613.
Gutiérrez Maldonado, J., Letosa Porta, A., RusCalafell, M., & Peñaloza Salazar, C. (2009). The assessment of attention deficit hyperactivity disorder in children using continous performance tasks in virtual environments. Anuario de Psicología, 40(2), 211–222.
Hale, J. B., Chen, S. A., Tan, S. C., Poon, K., Fitzer, K. R., & Boyd, L. A. (2016). Reconciling individual differences with collective needs: The juxtaposition of sociopolitical and neuroscience perspectives on remediation and compensation of student skill deficits. Trends in Neuroscience and Education, 5(2), 41–51.
Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128.
Hedges, L. V. (1984). Advances in statistical methods for meta-analysis. New Directions for Program Evaluation, 24, 25–42.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ: British Medical Journal, 327(7414), 557–560.
Huang-Pollock, C. L., Karalunas, S. L., Tam, H., & Moore, A. N. (2012). Evaluating vigilance deficits in ADHD: A meta-analysis of CPT performance. Journal of Abnormal Psychology, 121(2), 360.
Huedo-Medina, T. B., Sanchez-Meca, J., Marin-Martinez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods, 11, 193–206.
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.
Iriarte, Y., Diaz-Orueta, U., Cueto, E., Irazustabarrena, P., Banterla, F., & Climent, G. (2016). AULA—Advanced virtual reality tool for the assessment of attention: Normative study in Spain. Journal of Attention Disorders, 20(6), 542–568.
Kofler, M. J., Rapport, M. D., & Matt Alderson, R. (2008). Quantifying ADHD classroom inattentiveness, its moderators, and variability: a meta-analytic review. Journal of Child Psychology and Psychiatry, 49(1), 59–69.
Kofler, M. J., Raiker, J. S., Sarver, D. E., Wells, E. L., & Soto, E. F. (2016). Is hyperactivity ubiquitous in ADHD or dependent on environmental demands? Evidence from meta-analysis. Clinical Psychology Review, 46, 12–24.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
Losier, B. J., McGrath, P. J., & Klein, R. M. (1996). Error patterns on the continuous performance test in non-medicated and medicated samples of children with and without ADHD: A meta-analytic review. Journal of Child Psychology and Psychiatry, 37(8), 971–987.
Moreau, G., Guay, M., Achim, A., Rizzo, A., & Lageix, P. (2006). The virtual classroom: An ecological version of the continuous performance test–A pilot study. Annual Review of Cybertherapy and Telemedicine, 4, 59–66.
Mühlberger, A., Jekel, K., Probst, T., Schecklmann, M., Conzelmann, A., Andreatta, M., ... & Romanos, M. (2016). The influence of methylphenidate on hyperactivity and attention deficits in children with ADHD: A virtual classroom test. Journal of Attention Disorders, 1–13 https://doi.org/10.1177/1087054716647480.
Neguț, A., Matu, S. A., Sava, F. A., & David, D. (2016). Virtual reality measures in neuropsychological assessment: A meta-analytic review. The Clinical Neuropsychologist, 30(2), 165–184.
Nigg, J. T. (2005). Neuropsychologic theory and findings in attention-deficit/hyperactivity disorder: The state of the field and salient challenges for the coming decade. Biological Psychiatry, 57(11), 1424–1435.
Nolin, P., Martin, C., & Bouchard, S. (2009). Assessment of inhibition deficits with the virtual classroom in children with traumatic brain injury: a pilot-study. Annual Review of Cybertherapy and Telemedicine, 7, 240–242.
Nolin, P., Stipanicic, A., Henry, M., Joyal, C. C., & Allain, P. (2012). Virtual reality as a screening tool for sports concussion in adolescents. Brain Injury, 26(13–14), 1564–1573.
Nolin, P., Stipanicic, A., Henry, M., Lachapelle, Y., Lussier-Desrochers, D., & Allain, P. (2016). ClinicaVR: Classroom-CPT: A virtual reality tool for assessing attention and inhibition in children and adolescents. Computers in Human Behavior, 59, 327–333.
Ogles, B. M., Lunnen, K. M., & Bonesteel, K. (2001). Clinical significance: History, application, and current practice. Clinical Psychology Review, 21(3), 421–446.
Parsons, T. D., Bowerly, T., Buckwalter, J. G., & Rizzo, A. A. (2007). A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychology, 13(4), 363–381.
Parsons, T. D., Carlew, A. R., Magtoto, J., & Stonecipher, K. (2017). The potential of function-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychological Rehabilitation, 37(5), 777–807.
Parsons, T. D., & Rizzo, A. A. (2018). A virtual classroom for ecologically valid assessment of attention-deficit/hyperactivity disorder. In P. Sharkey (Ed.), Virtual reality Technologies for Health and Clinical Applications: Psychological and neurocognitive interventions (pp. ##- ##). Germany: Springer-Verlag.
Pelham, W. E., Waschbusch, D. A., Hoza, B., Gnagy, E. M., Greiner, A. R., Sams, S. E., … Carter, R. L. (2011). Music and video as distractors for boys with ADHD in the classroom: Comparison with controls, individual differences, and medication effects. Journal of Abnormal Child Psychology, 39(8), 1085–1098.
Pievsky, M. A., & McGrath, R. E. (2017). The neurocognitive profile of attention-deficit/hyperactivity disorder: A review of meta-analyses. Archives of Clinical Neuropsychology, 33(2), 143–157.
Pollak, Y., Shomaly, H. B., Weiss, P. L., Rizzo, A. A., & Gross-Tsur, V. (2010). Methylphenidate effect in children with ADHD can be measured by an ecologically valid continuous performance test embedded in virtual reality. CNS Spectrums, 15(2), 125–130.
Pollak, Y., Weiss, P. L., Rizzo, A. A., Weizer, M., Shriki, L., Shalev, R. S., & Gross-Tsur, V. (2009). The utility of a continuous performance test embedded in virtual reality in measuring ADHD-related deficits. Journal of Developmental & Behavioral Pediatrics, 30(1), 2–6.
Rapport, M. D., Chung, K. M., Shore, G., Denney, C. B., & Isaacs, P. (2000). Upgrading the science and technology of assessment and diagnosis: Laboratory and clinic-based assessment of children with ADHD. Journal of Clinical Child Psychology, 29(4), 555–568.
Review Manager (RevMan) [Computer program]. (2014) Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration.
Rizzo, A., Bowerly, T., Buckwalter, J., Klimchuk, D., Mitura, R., & Parsons, T. D. (2006). A virtual reality scenario for all seasons: The virtual classroom. CNS Spectrums, 11(1), 35–44.
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638.
Shaddish, W. R., & Haddock, C. K. (1994). Combining estimates of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 226–285). New York: Russell Sage Foundation.
Silverstein, M. L., Weinstein, M., & Turnbull, A. (2004). Nonpatient CPT performance varying target frequency and interstimulus interval on five response measures. Archives of Clinical Neuropsychology, 19(8), 1017–1025.
Sonuga-Barke, E. J., Sergeant, J. A., Nigg, J., & Willcutt, E. (2008). Executive dysfunction and delay aversion in attention deficit hyperactivity disorder: Nosologic and diagnostic implications. Child and Adolescent Psychiatric Clinics of North America, 17(2), 367–384.
Spooner, D. M., & Pachana, N. A. (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology, 21(4), 327–337.
U.S. Department of Education, (2007-2008) National Center for Education Statistics, Schools and Staffing Survey (SASS), "Public School Data File," 2007–08.
Willcutt, E. G., Doyle, A. E., Nigg, J. T., Faraone, S. V., & Pennington, B. F. (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57(11), 1336–1346.
Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225–240.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Parsons, T.D., Duffield, T. & Asbee, J. A Comparison of Virtual Reality Classroom Continuous Performance Tests to Traditional Continuous Performance Tests in Delineating ADHD: a Meta-Analysis. Neuropsychol Rev 29, 338–356 (2019). https://doi.org/10.1007/s11065-019-09407-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11065-019-09407-6