Abstract
This study examines (1) coalition survival, (2) prevalence of evidence-based prevention interventions (EBPIs) to reduce substance abuse implemented as part of the Tennessee Strategic Prevention Framework (SPF) State Incentive Grant (SIG), (3) EBPI sustainability, and (4) factors that predict EBPI sustainability. Secondary data were collected on 27 SPF SIG-funded coalitions and 88 EBPI and non-EBPI implementations. Primary data were collected by a telephone interview/web survey five and one-half years after the SPF SIG ended. Results from secondary data show that 25 of the 27 coalitions survived beyond the SPF SIG for one to five and one-half years; 19 coalitions (70%) were still active five and one-half years later. Further, 88 EBPIs and non-EBPIs were implemented by 27 county SPF SIG coalitions. Twenty-one (21) of 27 coalitions (78%) implemented one to three EBPIs, totaling 37 EBPI implementations. Based on primary survey data on 29 of the 37 EBPI implementations, 28 EBPIs (97%) were sustained between two and five and one-half years while 22 EBPI implementations (76%) were sustained for five and one-half years. When controlling for variability among coalitions (nesting of EBPIs in coalitions), increases in data resources (availability of five types of prevention data) was a strong predictor of length of EBPI sustainability. Positive change in extramural funding resources and level of expertise during SPF SIG implementation, as well as level of coalition formalization at the end of the SPF SIG predicted EBPI sustainability length. One intervention attribute (trialability) also predicted length of sustainability. Implications are discussed.
Similar content being viewed by others
References
Blanchet, K., & James, P. (2014). Can international health programmes be sustained after the end of international funding?: The case of eye care interventions in Ghana. BMC Health Services Research, 14 Retrieved from http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-77.
Bond, G. R., Drake, R. E., McHugo, G. J., Peterson, A. E., Jones, A. M., & Williams, J. (2014). Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 41, 228–236.
Bradford, N. K., Caffery, L. J., & Smith, A. C. (2016). Telehealth services in rural and remote Australia: A systematic review of models of care and factors influencing success and sustainability. Rural and Remote Health, 16. Retrieved from www.rrh.org.au/publishedarticles/article_print_3808.pdf.
Brown, L. D., Feinberg, M. E., & Greenberg, M. T. (2010). Determinants of community coalition ability to support evidence-based programs. Prevention Science, 11, 287–297.
Cernat, A., Couper, M. P., & Ofstedal, M. B. (2016). Estimation of mode effects in the health and retirement study using measurement models. Journal of Survey Statistics and Methodology, 4, 501–524.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.
Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.
Collins, D., Shamblen, S., Harris, M., Johnson, K., & Dwivedi, P. (2009). Evaluation of Tennessee SPF SIG local capacity building: Final report. Louisville: PIRE Retrieved from http://www.pire.org/documents/TN_Local_%20Capacity_%20Final_Report.pdf.
Cooper, B. R., Bumbarger, B. K., & Moore, J. E. (2015). Sustaining evidence-based prevention programs: Correlates in a large-scale dissemination initiative. Prevention Science, 16, 145–157.
De Leeuw, E. D., & Hox, J. J. (2015). Survey mode and mode effects. In U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, & P. Sturgis (Eds.), Improving survey methods: Lessons from recent research (pp. 22–34). New York: Routledge/Taylor & Francis.
Edwards, J. M., Stein-Seroussi, A., Flewelling, R. L., Orwin, R. G., & Zhang, L. (2015). Sustainability of state-level substance abuse prevention infrastructure after the completion of the SPF SIG. The Journal of Primary Prevention, 36, 177–186.
Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Implementing effective community-based prevention programs in the Community Youth Development Study. Youth Violence and Juvenile Justice, 6, 256–278.
Federal Register (2004). Notice of request for applications for Strategic Prevention Framework State Incentive Grants (SPF SIG) (SP 04–002). Retrieved from https://www.federalregister.gov/articles/2004/04/29/04-9656/notice-of-request-for-applications-for-strategic-prevention-framework-state-incentive-grants-spf-sig#h-12.
Feinberg, M. E., Bontempo, D. E., & Greenberg, M. T. (2008). Predictors and level of sustainability of community prevention coalitions. American Journal of Preventive Medicine, 34, 495–501.
Flay, B. R., Biglan, A., Boruch, R. F., Gonzalez Castro, F., Gottfredson, D., Kellam, S., …, Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175.
Flewelling, R. L., & Hanley, S. M. (2016). Assessing community coalition capacity and its association with underage drinking prevention effectiveness in the context of the SPF SIG. Prevention Science, 17, 830–840.
Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the Communities That are prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health, 51, 259–264.
Gloppen, K. M., Brown, E. C., Wagenaar, B. H., Hawkins, J. D., Rhew, I. C., & Oesterle, S. (2016). Sustaining adoption of science-based prevention through Communities That Care. Journal of Community Psychology, 44, 78–89.
Greenberg, M. T., Feinberg, M. E., Johnson, L. E., Perkins, D. F., Welsh, J. A., & Spoth, R. L. (2015). Factors that predict financial sustainability of community coalitions: Five years of findings from the PROSPER partnership project. Prevention Science, 16, 158–167.
IBM Corp. (2011). IBM SPSS Statistics for Windows, Version 20.0. Armonk: IBM Corp.
ICPSR (2016). Strategic Prevention Framework State Incentive Grant (SPF SIG) national cross-site evaluation. Retrieved from http://www.icpsr.umich.edu/icpsrweb/NAHDAP/studies/28921.
Ihaka, R., & Gentleman, R. (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5, 299–314.
Johnson, K., Hays, C., Center, H., & Daley, C. (2004). Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning, 27, 135–149.
Johnson, K., Collins, D., & Wandersman, A. (2013). Sustaining innovations in community prevention systems: A data-informed sustainability strategy. Journal of Community Psychology, 41, 322–340.
Lean, M., Leavey, G., Killaspy, H., Green, N., Harrison, I., Cook, S., Craig, T., Holloway, F., Arbuthnott, M., & King, M. (2015). Barriers to the sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: A qualitative study nested within a randomised controlled trial. BMC Psychiatry, 15. doi:10.1186/s12888-015-0592-9.
Livet, M., Courser, M., & Wandersman, A. (2008). The prevention delivery system: Organizational context and use of comprehensive programming frameworks. American Journal of Community Psychology, 41, 361–378.
Luke, D. A., Calhoun, A., Robichaux, C. B., Elliott, M. B., & Moreland-Russell, S. (2014). The program sustainability assessment tool: A new instrument for public health programs. Preventing Chronic Disease, 11, E12. doi:10.5888/pcd11.130184.
McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention, 40, 184–191.
Nargiso, J. E., Friend, K. B., Egan, C., Florin, P., Stevenson, J., Amodei, B., & Barovier, L. (2013). Coalitional capacities and environmental strategies to prevent underage drinking. American Journal of Community Psychology, 51, 222–231.
Pacific Institute for Research and Evaluation (PIRE). (2009). Strategies to reduce underage alcohol use: Typology and brief overview. Calverton: PIRE.
Peterson, A. E., Bond, G. R., Drake, R. E., McHugo, G. J., Jones, A. M., & Williams, J. R. (2014). Predicting the long-term sustainability of evidence-based practices in mental health care: An 8-year longitudinal analysis. Journal of Behavioral Health Services & Research, 41, 337–346.
Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., EISPACK authors, Heisterkamp, S., …, R-core (2016). nlme: Linear and nonlinear mixed effects models (Computer Software: v. 3.1–127). Retrieved from https://cran.r-project.org/web/packages/nlme.
Prevention Alliance of Tennessee (2015). PAT coalition portfolio. Retrieved from http://www.tncoalitions.org/coalitions.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., …, Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services, 38, 65–76.
Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free Press.
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.
Scaccia, J. P., Cook, B. S., Lamont, A., Wandersman, A., Castellow, J., Katz, J., & Beidas, R. S. (2015). A practical implementation science heuristic for organizational readiness: R=MC2. Journal of Community Psychology, 43, 484–501.
Scheirer, M. A. (2005). Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.
Scheirer, M. A., & Dearing, J. W. (2011). An agenda for research on the sustainability of public health programs. American Journal of Public Health, 101, 2059–2067.
Shults, R. A., Elder, R. W., Sleet, D. A., Nichols, J. L., Alao, M. O., Carande-Kulis, V., & Thompson, R. S. (2001). Reviews of evidence regarding interventions to reduce alcohol-impaired driving. American Journal of Preventive Medicine, 21, 66–88.
Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7. doi:10.1186/1748-5908-7-17.
Substance Abuse and Mental Health Services Administration (2016). National registry of evidence-based programs and practices. Retrieved from http://nrepp.samhsa.gov/01_landing.aspx.
Tennessee Department of Mental Health and Substance Abuse Services (2012). Prevention services provider directory, version 3. Retrieved from https://www.tn.gov/assets/entities/behavioral-health/sa/attachments/TPN-Prevention_Services_Directory_FY_13.pdf.
Tibbits, M. K., Bumbarger, B. K., Kyler, S., & Perkins, D. F. (2010). Sustaining evidence-based interventions under real world conditions: Results of a large-scale diffusion project. Prevention Science, 11, 252–262.
U.S. Department of Health and Human Services (2014). Performance improvement 2013–2014. Retrieved from https://aspe.hhs.gov/report/performance-improvement-2013-2014/what-are-program-outcomes-program-effectiveness-and-community-change-among-strategic-prevention-framework-state-incentive-grants.
Acknowledgments
This study was funded in part by the Pacific Institute for Research and Evaluation–Louisville Center and the University of South Carolina, Department of Psychology. We thank Leah Festa, the Director of the Prevention Alliance of Tennessee, the many former and current community coalition staff members, and the Tennessee Department of Mental Health and Substance Abuse Services staff for their assistance in providing data to the study. Chris Bayer, Louisville Center Production Manager, also provided important manuscript editing and formatting. A special thank you goes to the Prevention Science reviewers who provided valuable feedback on an earlier version of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This study was conducted using internal funding including in-kind funding from the Pacific Institute for Research and Evaluation and the University of South Carolina, Department of Psychology.
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
PIRE’s IRB reviewed the protocols for this study and deemed the study exempt under 45 CFR 46.101(b)(2). This study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Informed Consent
Written informed consent was not obtained from key respondents of our telephone interview/web-based survey conducted in 2015 since the PIRE IRB board deemed this study exempt under 45 CFR 46.101(b)(2). However, we did obtain verbal informed consent from the 2015–2016 sustainability survey participants.
Rights and permissions
About this article
Cite this article
Johnson, K., Collins, D., Shamblen, S. et al. Long-Term Sustainability of Evidence-Based Prevention Interventions and Community Coalitions Survival: a Five and One-Half Year Follow-up Study. Prev Sci 18, 610–621 (2017). https://doi.org/10.1007/s11121-017-0784-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-017-0784-2