Abstract
Implementation science in mental health is informed by other academic disciplines and industries. Conceptual and methodological territory charted in psychotherapy research is pertinent to two elements of the conceptual model of implementation posited by Aarons and colleagues (2010)—implementation fidelity and innovation feedback systems. Key characteristics of scientifically validated fidelity instruments, and of the feasibility of their use in routine care, are presented. The challenges of ensuring fidelity measurement methods are both effective (scientifically validated) and efficient (feasible and useful in routine care) are identified as are examples of implementation research attempting to balance these attributes of fidelity measurement.
Similar content being viewed by others
References
Aarons, G. A., Sommerfeld, D., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77, 270–280.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2010). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research.
AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC: AERA.
Baumann, B. L., Kolko, D. J., Collins, K., & Herschell, A. D. (2006). Understanding practitioners’ characteristics and perspectives prior to the dissemination of an evidence-based intervention. Child Abuse and Neglect, 30, 771–787.
Bearsley-Smith, C., Sellick, K., Chesters, J., Francis, K., & Gippsland Adolescent Depression Research Group. (2008). Treatment content in child and adolescent mental health services: Development of the treatment recording sheet. Administration and Policy Mental Health, 35, 423–435.
Cappella, E., Frazier, S. L., Atkins, M. S., Schoenwald, S. K., & Glisson, C. (2008). Enhancing schools’ capacity to support children in poverty: An ecological model of school-based mental health services. Administration and Policy In Mental Health, 35, 395–409.
Carroll, K. M., & Rounsaville, B. J. (2007). A vision of the next generation of behavioral therapies research in the addictions. Addiction, 102, 850–862.
Chamberlain, P. (2003). Treating chronic juvenile offenders: Advances made through the Oregon multidimensional treatment foster care model. Washington, DC: American Psychological Association.
Chamberlain, P., Price, J., Leve, L. D., Laurent, H., Landsverk, J. A., Reid, J. B., et al. (2008). Prevention of behavior problems for children in foster care: Outcomes and mediation effects. Prevention Science, 9, 17–27.
Chamberlain, P., Sprengelmeyer, P., Saldana, L., & Padget, C. (2010) Web-based observations to monitor treatment fidelity. Manuscript in preparation.
Child and Adolescent Mental Health Division (2003). Instructions and codebook for provider monthly summaries. Honolulu, HI: Hawaii Department of Health Child and Adolescent Mental Health Division. www.hawaii.gov/health/mental-health/camhd/resources/prov-agency/library/pdf/paf/paf-001.pdf. Accessed 23 March 2010.
Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566–579.
Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7, 5–20.
Fixsen, D. L, Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13.
Garland, A. F., Hawley, K. M., Brookman-Frazee, L., & Hurlburt, M. S. (2008a). Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 505–514.
Garland, A. F., McCabe, K. M., & Yeh, M. (2008b). Ethical challenges in practice-based mental health services research: Examples from research with children and families. Clinical Psychology: Science and Practice, 15, 118–124.
Hayes, S. C. (1998). Market-driven treatment development. The Behavior Therapist, 21, 32–33.
Heaton, K. J., Hill, C. E., & Edwards, L. A. (1995). Comparing molecular and molar methods of judging therapist techniques. Psychotherapy Research, 5, 141–153.
Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (2009). Multisystemic therapy for antisocial behavior in children and adolescents (2nd ed.). New York: The Guilford Press.
Hepner, K. A., Greenwood, G. L., Azocar, F., Miranda, J., & Burnam, M. A. (2009). Usual care psychotherapy for depression in a large managed behavioral health organization. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0247-6.
Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy, 33, 332–345.
Hurlburt, M. S., Garland, A. G., Nguyen, K., & Brookman-Frazee, L. (2009). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37, 230–243.
Kelley, S. D., Vides de Andrade, A. R., Sheffer, E., & Bickman, L. (2010). Exploring the black box: Measuring youth treatment process and progress in usual care. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-010-0298-8.
Malik, M. L., Beutler, L. E., Alimohamed, S., Gallagher-Thompson, D., & Thompson, L. (2003). Are all cognitive therapies alike? A comparison of cognitive and noncognitive therapy process and implications for the application of empirically supported treatments. Journal of Consulting and Clinical Psychology, 71, 150–158.
Manderscheid, R. W. (1998). From many to one: Addressing the crisis of quality in managed behavioral health care at the millennium. The Journal of Behavioral Health Services & Research, 25, 233–238.
McHugh, R. K., Murray, H. W., & Barlow, D. H. (2009). Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: The promise of transdiagnostic interventions. Behaviour Research and Therapy, 47(11), 946–953.
Mihalic, S. (2004). The importance of implementation fidelity. Emotional and Behavioral Disorders in Youth, 4(83–86), 99–105.
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.
National Institute of Mental Health. (2006). The road ahead: Research partnerships to transform services. A report by the National Advisory Mental Health Council’s Workgroup on Services and Clinical Epidemiology Research. Rockville, MD: National Institutes of Health, Department of Health and Human Services.
Orimoto, T. E., Higa-McMillan, C. K., Mueller, C., & Tolman, R. T. (2009, February). Organization of therapeutic practices in treatment as usual. Paper presented at the 22nd annual research conference of the Research and Training Center for Children’s Mental Health, Tampa, FL.
Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841.
Real, K., & Poole, M. S. (2005). Innovation implementation: Conceptualization and measurement in organizational research. Research in Organizational Change and Development, 15, 63–134.
Rossi, P. H. (1978). Issues in the evaluation of human services delivery. Evaluation Quarterly, 2(4), 573–599.
Schoenwald, S. K. (2008). Toward evidence-based transport of evidence-based treatments: MST as an example. Journal of Child and Adolescent Substance Abuse, 17(3), 69–91.
Schoenwald, S. K., & Henggeler, S. W. (2004). A public health perspective on the transport of evidence based practices. Clinical Science and Practice, 11, 360–363.
Schoenwald, S. K., Henggeler, S. W., Brondino, M. J., & Rowland, M. D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process, 39, 83–103.
Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., et al. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy in Mental Health and Mental Health Services Research, 35, 84–97.
Sheidow, A. J., Donohue, B. C., Hill, H. H., Henggeler, S. W., & Ford, J. D. (2008). Development of an audio-tape review system for supporting adherence to an evidence-based treatment. Professional Psychology: Research and Practice, 39, 553–560.
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. E. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620–630.
Weersing, V. R., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child Psychology, 31, 168–180.
Young, J., Daleiden, E. L., Chorpita, B. F., Schiffman, S., & Mueller, C. W. (2007). Assessing stability between treatment planning documents in a system of care. Administration and Policy in Mental Health and Mental Health Services Research, 34, 530–539.
Acknowledgments
The primary support for this manuscript was provided by NIMH research grants 1P30MH074678 (J. Landsverk, PI) and 1P20MH0784458 (M. Atkins, PI). The authors thank the Implementation Methods Research Group (NIMH grant 1P30MH074678) Executive Committee for helpful reviews of manuscript drafts that lead to the inclusion of Table 2 and Figure 1; Marc Atkins, Bruce Chorpita, and David Henry for their participation in Think Tank discussions pertinent to the manuscript; and, V. Robin Weersing.
Author information
Authors and Affiliations
Corresponding author
Additional information
Author Note
Sonja K. Schoenwald is a Board Member and stockholder in MST Services, LLC, which has the exclusive licensing agreement through the Medical University of South Carolina for the dissemination of MST technology.
Rights and permissions
About this article
Cite this article
Schoenwald, S.K., Garland, A.F., Chapman, J.E. et al. Toward the Effective and Efficient Measurement of Implementation Fidelity. Adm Policy Ment Health 38, 32–43 (2011). https://doi.org/10.1007/s10488-010-0321-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-010-0321-0