Abstract
The study examined the confidence accuracy relationship, and the influence of child characteristics on clinician confidence, when predicting a diagnosis of Autism Spectrum Disorder during screening of 125 referred children aged under 3.5 years. The diagnostic process included observation, interview, language and developmental testing. Clinical judgement accuracy was compared against final diagnosis for high and low confidence levels (with confidence assessed on a 0–100 % scale). We identified a significant CA relationship with predictive accuracy highest at confidence levels of 90–100 %. Parent report of unusual behaviors was the only significant independent predictor of confidence. Clinicians’ confidence may be important when evaluating decisions to refer, or not to refer, children for further diagnostic assessment.
Similar content being viewed by others
Notes
This may occur during cognitive processing. For example, Insabato et al. (2010) pose a two-stage decision mechanism where a second decision is based on confidence associated with the previous decision. More pragmatically, future decisions by other clinicians may also be influenced by earlier judgements, clinical notes, recommendations, etc.
Intellectual and developmental disability clinic (n = 15); behavioral health services (n = 7); psychological assessment (n = 1); speech services (n = 2); neurodevelopmental clinic (n = 1); physical therapy (n = 1).
We were able to assess 11 of the 33 children not referred for a diagnostic assessment with either the ADOS-2 or ADI-R. Four children did not meet ASD criteria on the ADI-R, five were under cut-off for ASD on the ADOS-2 and two children were in the low risk range but above cut-off for Autism Spectrum. Of the children who met ASD criteria on the ADOS-2, one had been assessed in the intellectual and developmental delay clinic and received a diagnosis of a language disorder and disruptive behavioral disorder; the other was not assessed further.
References
Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA Preschool forms and profiles. Burlington, VT: University of Vermont Department of Psychiatry.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Publishing.
Barbaro, J., Ridgway, L., & Dissanayake, C. (2011). Developmental surveillance of infants and toddlers by maternal and child health nurses in an Australian community-based setting: Promoting the early identification of Autism Spectrum Disorders. Journal of Pediatric Nursing, 26, 334–347.
Bauminger, N., Solomon, M., & Rogers, S. J. (2010). Internalizing and externalizing behaviors in ASD. Autism Research, 3, 101–112.
Brewer, N. (2006). Uses and abuses of eyewitness confidence. Legal and Criminological Psychology, 11, 3–23.
Brewer, N., & Wells, G. (2006). The confidence-accuracy relationship in eyewitness identification: Effects of lineup instructions, foil similarity, and target-absent base rates. Journal of Experimental Psychology: Applied, 12, 11–30.
Carlin, A. S., & Hewitt, P. L. (1990). The discrimination of patient generated and randomly generated MMPIs. Journal of Personality Assessment, 54, 24–29.
Centers for Disease Control and Prevention. (2012). Prevalence of autism spectrum disorders—Autism and Developmental Disabilities Monitoring Network, 14 sites, United States, 2008. MMWR 2012; 61 (No. SS-3).
Charman, T., & Baird, G. (2002). Practitioner review: Diagnosis of autism spectrum disorder in 2-and 3-year-old children. Journal of Child Psychology and Psychiatry, 43, 289–305.
Corsello, C., Akshoomoff, N., & Stahmer, A. (2013). Diagnosis of autism spectrum disorders in 2-year-olds: A study of community practice. Journal of Child Psychology and Psychiatry, 54, 178–185.
Dawson, G., Jones, E. J. H., Merkle, K., Venema, K., Lowy, R., Faja, S., & Webb, S. J. (2012). Early behavioral intervention is associated with normalized brain activity in young children with autism. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 1150–1159.
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., & Varley, J. (2010). Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics, 125, 17–23.
Goldstein, S., & Naglieri, J. A. (2010). Autism Spectrum Rating Scales. North Tonawanda, NY: Multi Health Systems.
Guthrie, W., Swineford, L., Nottke, C., & Wetherby, A. (2013). Early diagnosis of autism spectrum disorder: Stability and change in clinical diagnosis and symptom presentation. Journal of Child Psychology and Psychiatry, 54, 582–590.
Hedley, D., Nevill, R. E., Monroy Moreno, Y., Fields, N., Wilkins, J., Butter, E., & Mulick, J. A. (2015). Efficacy of the ADEC in identifying Autism Spectrum Disorder in clinically referred toddlers in the US. Journal of Autism and Developmental Disorders, 45, 2337–2348.
Insabato, A., Pannunzi, M., Rolls, E. T., & Deco, G. (2010). Confidence-related decision making. Journal of Neurophysiology, 104, 539–547.
Levy, S. E., Giarelli, E., Lee, L. C., Schieve, L. A., Kirby, R. S., Cunniff, C., & Rice, C. E. (2010). Autism spectrum disorder and co-occurring developmental, psychiatric, and medical conditions among children in multiple populations of the United States. Journal of Developmental and Behavioral Pediatrics, 31, 267–275.
Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20, 159–183.
Lichtenstein, S., Fischhoff, B., & Phillips, L. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slavic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.
Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012a). Autism diagnostic observation schedule, Second Edition (ADOS-2) Manual (Part II): Toddler Module. Torrance, CA: Western Psychological Services.
Lord, C., Risi, S., DiLavore, P. S., Shulman, C., Thurm, A., & Pickles, A. (2006). Autism from 2 to 9 years of age. Archives of General Psychiatry, 63, 694–701.
Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012b). Autism diagnostic observation schedule, Second Edition (ADOS-2) Manual (Part I): Modules 1–4. Torrance, CA: Western Psychological Services.
Messinger, D., Young, G. S., Ozonoff, S., Dobkins, K., Carter, A., Zwaigenbaum, L., & Sigman, M. (2013). Beyond autism: A baby siblings research consortium study of high-risk children at three years of age. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 300–308.
Miller, D. J., Spengler, E. S., & Spengler, P. M. (2015). A meta-analysis of confidence and judgment accuracy in clinical decision making. Journal of Counselling Psychology, 62, 553–567.
Mullen, E. M. (1995). Mullen scales of early learning (AGS ed.). Circle Pines, MN: American Guidance Service Inc.
Murphy, A. H., & Winkler, R. L. (1974). Credible interval temperature forecasting: Some experimental results. Monthly Weather Review, 102, 784–794.
National Autism Center. (2015). Findings and conclusions: National Standard Project, Phase 2: Addressing the need for evidence-based practice guidelines for Autism Spectrum Disorder. Randolph, MA: National Autism Center.
Oskamp, S. (1965). Overconfidence in case-study judgements. Journal of Consulting Psychology, 29, 261–265.
Ozonoff, S., Heung, K., Byrd, R., Hansen, R., & Hertz-Picciotto, I. (2008). The onset of Autism: Patterns of symptom emergence in the first years of life. Autism Research, 1, 320–328.
Ozonoff, S., Young, G. S., Steinfeld, M. B., Hill, M. M., Cook, I., Hutman, T., & Sigman, M. (2009). How early do parent concerns predict later Autism diagnosis? Journal of Developmental and Behavioral Pediatrics, 30, 367–375.
Ronis, D. L., & Yates, J. F. (1987). Components of probability judgement accuracy: Individual consistency and effects of subject matter and assessment method. Organizational Behavior and Human Decision Processes, 40, 193–218.
Shimabukuro, T. T., Grosse, S. D., & Rice, C. (2008). Medical expenditures for children with an autism spectrum disorder in a privately insured population. Journal of Autism and Developmental Disorders, 38, 546–552.
Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., & Baird, G. (2008). Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 921–929.
Sparrow, S., Cicchetti, D., & Balla, D. (2005). Vineland adaptive behavior scales (2nd ed.). Minneapolis, MN: Pearson Assessment.
Wiggins, L. D., Baio, J., & Rice, C. (2006). Examination of the time between first evaluation and first Autism Spectrum Diagnosis in a population-based sample. Developmental and Behavioral Pediatrics, 27, 79–87.
Wolff, J., Botteron, K. N., Dager, S. R., Elison, J. T., Estes, A. M., Gu, H., & Piven, J. (2014). Longitudinal patterns of repetitive behavior in toddlers with autism. Journal of Child Psychology and Psychiatry, 55, 945–953.
Young, R. (2007). Autism detection in early childhood: ADEC. Camberwell, VIC: ACER Press.
Acknowledgments
This research was supported by Nationwide Children’s Hospital Clinical and Translational Intramural Grant #203213 awarded to Darren Hedley. We would like to thank the staff and families who participated in the study. We also thank Sarah Beinkampen, Natalie Fields, Mark Jones, Emily Mariotti, Yessica Monroy Moreno and Brianna Murphy for their assistance with data collection, and Dr. Elizabeth Kryszak and Dr. Nathan Weber for their contributions to the study design.
Author Contributions
DH, NB and JAM conceived and designed the study. DH and RN collected the data. DH, NB, RN and MU performed the analysis. All authors contributed to the writing and review of the manuscript. All authors approved the final version.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors confirm that there are no conflict of interest to declare.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study where required to do so by the Institutional Review Board.
Appendices
Appendix 1
Questionnaire
We would like to know your view regarding your clinical impression of the child you assessed. Please take a few minutes to answer the following questions immediately following the clinical interview.
Please circle one response for the following questions:
1a) Based on the information available do you think the child has ASD?
YES (answer 1b only) | NO (answer 1c only) |
1b) If you think the child has ASD, how confident are you about this?
0 % | 10 % | 20 % | 30 % | 40 % | 50 % | 60 % | 70 % | 80 % | 90 % | 100 % |
Not confident at all | Absolutely certain |
1c) If you think the child does not have ASD, how confident are you about this?
0 % | 10 % | 20 % | 30 % | 40 % | 50 % | 60 % | 70 % | 80 % | 90 % | 100 % |
Not confident at all | Absolutely certain |
Appendix 2
See Table 4.
Rights and permissions
About this article
Cite this article
Hedley, D., Brewer, N., Nevill, R. et al. The Relationship Between Clinicians’ Confidence and Accuracy, and the Influence of Child Characteristics, in the Screening of Autism Spectrum Disorder. J Autism Dev Disord 46, 2340–2348 (2016). https://doi.org/10.1007/s10803-016-2766-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10803-016-2766-9