Skip to main content
SpringerLink
Log in

A Clinician’s Guide to Machine Learning in Neuropsychological Research and Practice

  • Published:
Journal of Pediatric Neuropsychology Aims and scope Submit manuscript

Abstract

Machine learning (ML) techniques can help harness insights from data that complement and extend those that can be attained by traditional statistical methods. The current article introduces clinicians to concepts underlying ML and explores how it can be applied within the domain of neuropsychology. Specifically, we illustrate an application of ML to a dataset that includes a battery of standardized measures designed to provide diagnostic support for concussions, including standardized neurocognitive (CPT 3) and neurobehavioral (BESS, NIH 4 meter gait) measures, gait sensor data, and a CDC concussion symptom checklist. These variables were used to predict the decision-making of a pediatric neurologist evaluating a group of child/adolescent patients. With a sample of 111 cases, ML (using a general linear model and deep learning as illustrations) achieved accuracies of 91% and 84.8% and AUCs of 1.0 and .947, respectively, when predicting the neurologist’s binomial decision-making (safe/remove). In presenting the data and various considerations for interpretation, we attempt to balance both the promise and perils of ML.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In the context of artificial neural networks, a rectifier is an activation function defined as the positive part of its argument:

    f(x) = x+ = max(0, x),

    where x is the input to a neuron. Wikipedia

  2. Softmax is a function that takes as input a vector of K real numbers, and normalizes it into a probability distribution consisting of K probabilities proportional to the exponentials of the input numbers. Wikipedia

References

  • Battista, P., Salvatore, C., & Castiglioni, I. (2017, 2017). Optimizing neuropsychological assessments for cognitive, behavioral, and functional impairment classification: a machine learning study. Behavioural Neurology, 1–19. https://doi.org/10.1155/2017/1850909.

    Article  Google Scholar 

  • Bleiberg, J., Cernich, A. N., Cameron, K., et al. (2004). Duration of cognitive impairment after sports concussion. Neurosurgery, 54, 1073–1080. https://doi.org/10.1227/01.NEU.0000118820.33396.6A.

    Article  PubMed  Google Scholar 

  • Broglio, S.P., Ferrara, M.S., Piland, S.G., Anderson, R.B. (2006). Concussion history is not a predictor of computerised neurocognitive performance. British Journal of Sports Medicine, 40(9), 802-805.

    Article  Google Scholar 

  • Broglio, S. P., & Puetz, T. W. (2008). The effect of sport concussion on neurocognitive function, self-report symptoms, and postural control: a meta-analysis. Sports Medicine, 38, 53–67.

    Article  Google Scholar 

  • Broglio, S. P., Ferrara, M. S., Macciocchi, S. N., Baumgartner, T. A., & Elliott, R. (2007). Test-retest reliability of computerized concussion assessment programs. Journal of Athletic Training, 42(4), 509–514.

    PubMed  PubMed Central  Google Scholar 

  • Bruce, J., Echemendia, R., Meeuwisse, W., Comper, P., & Sisco, A. (2014). 1-year test–retest reliability of ImPACT in professional ice hockey players. The Clinical Neuropsychologist, 28(1), 14–25.

    Article  Google Scholar 

  • Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.

    Article  Google Scholar 

  • Frank, G. (1984). The Boulder Model: History, rationale, and critique. Professional Psychology: Research and Practice, 15(3), 417–435. https://doi.org/10.1037/0735-7028.15.3.417.

    Article  Google Scholar 

  • Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: a meta-analysis. Psychological Assessment, 12, 19–30. https://doi.org/10.1037//1040-3590.12.1.19.

    Article  PubMed  Google Scholar 

  • Henry, L. C., Elbin, R. J., Collins, M. W., et al. (2016). Examining recovery trajectories after sport-related concussion with a multimodal clinical assessment approach. Neurosurgery, 78, 232–241. https://doi.org/10.1227/NEU.0000000000001041.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hinton, G. (2018). Deep Learning—a technology with the potential to transform health care. Journal of the American Medical Association, 320(11), 1101–1102. https://doi.org/10.1001/jama.2018.11100.

    Article  PubMed  Google Scholar 

  • Iverson, G. L., Gardner, A. J., Terry, D. P., et al. (2017). Predictors of clinical recovery from concussion: a systematic review. British Journal of Sports Medicine, 51, 941–948.

    Article  Google Scholar 

  • Kaye, A. J., Gallagher, R., Callahan, J. M., & Nance, M. L. (2010). Mild traumatic brain injury in the pediatric population: the role of the pediatrician in routine follow-up. Journal of Trauma, 68, 1396–1400.

    Article  Google Scholar 

  • Kirelik, S. B., & McAvoy, K. (2016). Acute concussion management with remove-reduce/educate/adjust-accommodate/pace (REAP). The Journal of emergency medicine, 50(2), 320–324.

    Article  Google Scholar 

  • Lau, B., Lovell, M. R., Collins, M. W., & Pardini, J. (2009). Neurocognitive and symptom predictors of recovery in high school athletes. Clinical Journal of Sport Medicine, 19(3), 216–221.

    Article  Google Scholar 

  • Lecci, L., Wiiliams, M., Taravath, S., Frank, H.G., Dugan, K., Page, G.R, & Keith, J.R. (in press). Validation of a concussion screening battery for use in medical settings. Archives of Clinical Neuropsychology.

  • LeCun, Y., Bengio, Y., & Hinton, G. H. (2015). Deep learning. Nature, 521, 436–444. https://doi.org/10.1038/nature14539.

    Article  PubMed  Google Scholar 

  • Macciocchi, S. N., Barth, J. T., Alves, W., et al. (1996). Neuropsychological functioning and recovery after mild head injury in collegiate Athletes. Neurosurgery, 39, 510–514.

    Article  Google Scholar 

  • McCrea, M., Guskiewicz, K. M., Marshall, S. W., et al. (2003). Acute effects and recovery time following concussion in collegiate football players: The NCAA Concussion Study. JAMA, 290, 2556–2563. https://doi.org/10.1001/jama.290.19.2556.

    Article  PubMed  Google Scholar 

  • McCrory, P., Meeuwisse, W. H., Aubry, M., Cantu, R. C., Dvorak, J., Echemendia, R. J., et al. (2013). Consensus statement on concussion in sport—the 4th International Conference on Concussion in Sport held in Zurich, November 2012. PM&R, 5(4), 255–279.

    Article  Google Scholar 

  • McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., et al. (2017). Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51(11), 838–847.

    PubMed  Google Scholar 

  • Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence (Vol. x 149 pp.). Minneapolis: University of Minnesota Press. https://doi.org/10.1037/11281-000.

    Book  Google Scholar 

  • Parker, T. M., Osternig, L. R., van Donkelaar, P., & Chou, L. S. (2006). Gait stability after a concussion. Medicine and Science in Sports & Exercise, 38, 1031–1040.

    Article  Google Scholar 

  • Resch, J. E., Macciocchi, S., & Ferrara, M. S. (2013). Preliminary evidence of equivalence of alternate forms of the ImPACT. The Clinical Neuropsychologist, 27(8), 1265–1280.

    Article  Google Scholar 

  • Rosenblatt, F. (1957). The perceptron – a perceiving and recognizing automation. Report 85-460-1, Cornell Aeronautical Laboratory.

  • Sahl, S. M. (2015). Estimating R 2 shrinkage in regression. International Journal of Technical Research and Applications, 3, 01–06.

    Google Scholar 

  • Schatz, P., & Putz, B. O. (2006). Cross-validation of measures used for computer-based assessment of concussion. Applied Neuropsychology, 13(3), 151–159.

    Article  Google Scholar 

  • Shatte, A., Hutchinson, D., & Teague, S. (2019). Machine learning in mental health: a scoping review of methods and applications. Psychological Medicine, 49(9), 1426–1448. https://doi.org/10.1017/S0033291719000151.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of North Carolina Wilmington, 601 South College Rd., Box 5612, Wilmington, NC, 28403-5612, USA

    Julian Keith & Len Lecci

  2. Internal Medicine, New Hanover Regional Medical Center, Wilmington, NC, USA

    Mark Williams

  3. Pediatric Neurology, New Hanover Regional Medical Center, Wilmington, NC, USA

    Sasidharan Taravath

Authors
  1. Julian Keith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sasidharan Taravath
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Len Lecci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Len Lecci.

Ethics declarations

Conflict of Interest

Julian Keith is a paid consultant for SportGait, which is the concussion battery utilized in the current research. Mark Williams is one of the founders and stockholder in SportGait. Len Lecci is a paid consultant and stockholder in SportGait. Sasiharan Taravath has no conflicts to report and he provided the clinical data herein reported.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keith, J., Williams, M., Taravath, S. et al. A Clinician’s Guide to Machine Learning in Neuropsychological Research and Practice. J Pediatr Neuropsychol 5, 177–187 (2019). https://doi.org/10.1007/s40817-019-00075-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40817-019-00075-1

Keywords

Search

Navigation