Skip to main content

Advertisement

SpringerLink
Log in

Engaging multilevel stakeholders in an implementation trial of evidence-based quality improvement in VA women’s health primary care

  • Original Research
  • Published:
Translational Behavioral Medicine

Abstract

The Veterans Health Administration (VHA) has undertaken primary care transformation based on patient-centered medical home (PCMH) tenets. VHA PCMH models are designed for the predominantly male Veteran population, and require tailoring to meet women Veterans’ needs. We used evidence-based quality improvement (EBQI), a stakeholder-driven implementation strategy, in a cluster randomized controlled trial across 12 sites (eight EBQI, four control) that are members of a Practice-Based Research Network. EBQI involves engaging multilevel, inter-professional leaders and staff as stakeholders in reviewing evidence and setting QI priorities. The goal of this analysis was to examine processes of engaging stakeholders in early implementation of EBQI to tailor VHA’s medical home for women. Four inter-professional regional stakeholder planning meetings were conducted; these meetings engaged stakeholders by providing regional data about gender disparities in Veterans’ care experiences. Subsequent to each meeting, qualitative interviews were conducted with 87 key stakeholders (leaders and staff). Stakeholders were asked to describe QI efforts and the use of data to change aspects of care, including women’s health care. Interview transcripts were summarized and coded using a hybrid deductive/inductive analytic approach. The presentation of regional-level data about gender disparities resulted in heightened awareness and stakeholder buy-in and decision-making related to women’s health-focused QI. Interviews revealed that stakeholders were familiar with QI, with regional and facility leaders aware of inter-disciplinary committees and efforts to foster organizational change, including PCMH transformation. These efforts did not typically focus on women’s health, though some informal efforts had been undertaken. Barriers to engaging in QI included lack of communication across clinical service lines, fluidity in staffing, and lack of protected time. Inter-professional, multilevel stakeholders need to be engaged in implementation early, with data and discussion that convey the importance and relevance of a new initiative. Stakeholder perspectives on institutional norms (e.g., gender norms) and readiness for population-specific QI are useful drivers of clinical initiatives designed to transform care for clinical subpopulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. National Center for Veterans Analysis and Statistics. Projected Veteran population 2013 to 2043. October 13, 2014. Available at: http://www.va.gov/vetdata/docs/quickfacts/population_slideshow.pdf. Accessibility verified December 31, 2016.

  2. de Kleijn, M., Lagro-Janssen, A. L., Canelo, I., & Yano, E. M. (2015). Creating a roadmap for delivering gender-sensitive comprehensive care for women Veterans: results of a national expert panel. Medical Care, 53, S156–S164.

    Article  Google Scholar 

  3. Rosland, A. M., Nelson, K., Sun, H., et al. (2013). The patient-centered medical home in the Veterans Health Administration. The American Journal of Managed Care, 19, e263–e272.

    PubMed  Google Scholar 

  4. Yano, E. M., Bair, M. J., Carrasquillo, O., Krein, S. L., & Rubenstein, L. V. (2014). Patient Aligned Care Teams (PACT): VA’s journey to implement patient-centered medical homes. Journal of General Internal Medicine, 29, S547–S549.

    Article  PubMed  Google Scholar 

  5. O’Toole, T. P., Johnson, E. E., Aiello, R., Kane, V., & Pape, L. (2016). Tailoring care to vulnerable populations by incorporating social determinants of health: the Veterans Health Administration’s “Homeless Patient Aligned Care Team” program. Preventing Chronic Disease, 13, E44.

    PubMed  PubMed Central  Google Scholar 

  6. Hernandez, S. E., Taylor, L., Grembowski, D., et al. (2016). A first look at PCMH implementation for minority Veterans: room for improvement. Medical Care, 54, 253–261.

    Article  PubMed  Google Scholar 

  7. Engel, P. A., Spencer, J., Paul, T., & Boardman, J. B. (2016). The geriatrics in primary care demonstration: integrating comprehensive geriatric care into the medical home: preliminary data. Journal of the American Geriatrics Society, 64, 875–879.

    Article  PubMed  Google Scholar 

  8. Yano, E. M., Haskell, S., & Hayes, P. (2014). Delivery of gender-sensitive comprehensive primary care to women veterans: implications for VA Patient Aligned Care Teams. Journal of General Internal Medicine, 29, S703–S707.

    Article  PubMed  Google Scholar 

  9. National Center for Veterans Analysis and Statistics. Profile of women Veterans: 2015. December, 2016. Available at: https://www.va.gov/vetdata/docs/SpecialReports/Women_Veterans_Profile_12_22_2016.pdf. Accessibility verified December 31, 2016.

  10. Wilson LC. The prevalence of military sexual trauma: a meta-analysis. Trauma Viol Abuse, first published date: December 16, 2016.

  11. Mattocks, K. M. (2015). Care coordination for women veterans: bridging the gap between systems of care. Medical Care, 53, S8–S9.

    Article  PubMed  Google Scholar 

  12. Chuang E, Brunner J, Mak S, et al. Challenges with implementing a patient-centered medical home model for women Veterans. Women’s Health Issues, 2017.

  13. Hamilton, A. B., Frayne, S. M., Cordasco, K. M., & Washington, D. L. (2013). Factors related to attrition from VA healthcare use: findings from the National Survey of women Veterans. Journal of General Internal Medicine, 28, S510–S516.

    Article  PubMed  Google Scholar 

  14. Kirchner, J. E., Parker, L. E., Bonner, L. M., Fickel, J. J., Yano, E. M., & Ritchie, M. J. (2012). Roles of managers, frontline staff and local champions, in implementing quality improvement: stakeholders’ perspectives. Journal of Evaluation in Clinical Practice, 18, 63–69.

    Article  PubMed  Google Scholar 

  15. Yano, E. M., Green, L. W., Glanz, K., et al. (2012). Implementation and spread of interventions into the multilevel context of routine practice and policy: implications for the cancer care continuum. Journal of the National Cancer Institute. Monographs, 2012, 86–99.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Solimeo, S. L., Stewart, K. R., Stewart, G. L., & Rosenthal, G. (2014). Implementing a patient centered medical home in the Veterans health administration: perspectives of primary care providers. Healthc (Amst), 2, 245–250.

    Article  Google Scholar 

  17. Rubenstein, L. V., Chaney, E. F., Ober, S., et al. (2010). Using evidence-based quality improvement methods for translating depression collaborative care research into practice. Families, Systems & Health, 28, 91–113.

    Article  Google Scholar 

  18. Curran, G. M., Pyne, J., Fortney, J. C., et al. (2011). Development and implementation of collaborative care for depression in HIV clinics. AIDS Care, 23, 1626–1636.

    Article  PubMed  Google Scholar 

  19. Fortney, J., Enderle, M., McDougall, S., et al. (2012). Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implementation Science, 11, 30.

    Article  Google Scholar 

  20. Hamilton, A. B., Cohen, A. N., Glover, D. L., et al. (2013). Implementation of evidence-based employment services in specialty mental health. Health Services Research, 48, 2224–2244.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Rubenstein, L. V., Stockdale, S. E., Sapir, N., et al. (2014). A patient-centered primary care practice approach using evidence-based quality improvement: rationale, methods, and early assessment of implementation. Journal of General Internal Medicine, 29, S589–S597.

    Article  PubMed  Google Scholar 

  22. Yoon, J., Chow, A., & Rubenstein, L. V. (2016). Impact of medical home implementation through evidence-based quality improvement on utilization and costs. Medical Care, 54, 118–125.

    Article  PubMed  Google Scholar 

  23. Nadeem, E., Olin, S. S., Hill, L. C., Hoagwood, K. E., & Horwitz, S. M. (2013). Understanding the components of quality improvement collaboratives: a systematic literature review. The Milbank Quarterly, 91, 354–394.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Yano, E. M., Darling, J. E., Hamilton, A. B., et al. (2016). Cluster randomized trial of a multilevel evidence-based quality improvement approach to tailoring VA Patient Aligned Care Teams to the needs of women Veterans. Implementation Science, 11, 101.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Frayne, S. M., Carney, D. V., Bastian, L., et al. (2013). The VA Women’s Health Practice-Based Research Network: amplifying women veterans’ voices in VA research. Journal of General Internal Medicine, 28, S504–S509.

    Article  PubMed  Google Scholar 

  26. Kwan, B. M., Sills, M. R., Graham, D., et al. (2016). Stakeholder engagement in a patient-reported outcomes (PRO) measure implementation: a report from the SAFTINet Practice-Based Research Network (PBRN). Journal of American Board of Family Medicine, 29, 102–115.

    Article  Google Scholar 

  27. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217–226.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Stetler, C. B., Legro, M. W., Wallace, C. M., et al. (2006). The role of formative evaluation in implementation research and the QUERI experience. Journal of General Internal Medicine, 21, S1–S8.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Rubenstein, L. V., Danz, M. S., Crain, A. L., Glasgow, R. E., Whitebird, R. R., & Solberg, L. I. (2014). Assessing organizational readiness for depression care quality improvement: relative commitment and implementation capability. Implementation Science, 9, 173.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Concannon, T. W., Meissner, P., Grunbaum, J. A., et al. (2012). A new taxonomy for stakeholder engagement in patient-centered outcomes research. Journal of General Internal Medicine, 27, 985–991.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Patton, M. Q. (2002). Qualitative Research and Evaluation Methods (3rd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  32. Averill, J. B. (2002). Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qualitative Health Research, 12, 855–866.

    Article  PubMed  Google Scholar 

  33. Miles, M. B., & Huberman, A. M. (1994). Qualitative Data Analysis: an Expanded Sourcebook. Thousand Oaks, CA: Sage.

    Google Scholar 

  34. Scoville, R., & Little, K. (2014). Comparing Lean and Quality Improvement. IHI White Paper. Cambridge, Massachusetts: Institute for Healthcare Improvement (Available at ihi.org).

    Google Scholar 

  35. Smith, J. L., Williams Jr., J. W., Owen, R. R., Rubenstein, L. V., & Chaney, E. (2008). Developing a national dissemination plan for collaborative care for depression: QUERI series. Implementation Science, 3, 59.

    Article  PubMed  PubMed Central  Google Scholar 

  36. McCormack, B., Rycroft-Malone, J., Decorby, K., et al. (2013). A realist review of interventions and strategies to promote evidence-informed healthcare: a focus on change agency. Implementation Science, 8, 107.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Mendel, P., Ngo, V. K., Dixon, E., et al. (2011). Partnered evaluation of a community engagement intervention: use of a kickoff conference in a randomized trial for depression care improvement in underserved communities. Ethnicity & Disease, 21, S1 78-88.

    Google Scholar 

  38. Chen, Q., Shin, M. H., Chan, J. A., et al. (2016). Partnering with VA stakeholders to develop a comprehensive patient safety data display: lessons learned from the field. American Journal of Medical Quality, 31, 178–186.

    Article  PubMed  Google Scholar 

  39. Salyers, M. P., Godfrey, J. L., McGuire, A. B., Gearhart, T., Rollins, A. L., & Boyle, C. (2009). Implementing the illness management and recovery program for consumers with severe mental illness. Psychiatric Services, 60, 483–490.

    Article  PubMed  Google Scholar 

  40. Ahmad, R., Kyratsis, Y., & Holmes, A. (2012). When the user is not the chooser: learning from stakeholder involvement in technology adoption decisions in infection control. The Journal of Hospital Infection, 81, 163–168.

    Article  CAS  PubMed  Google Scholar 

  41. Berry, C. A., Mijanovich, T., Albert, S., et al. (2013). Patient-centered medical home among small urban practices serving low-income and disadvantaged patients. Annals of Family Medicine, 11, S82–S89.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Martinson, B. C., Nelson, D., Hagel-Campbell, E., et al. (2016). Initial results from the Survey of Organizational Research Climates (SOuRCe) in the U.S. Department of Veterans Affairs Healthcare System. PloS One, 11, e0151571.

    Article  PubMed  PubMed Central  Google Scholar 

  43. MacGregor, C., Hamilton, A. B., Oishi, S. M., & Yano, E. M. (2011). Description, development, and philosophies of mental health service delivery for female veterans in the VA: a qualitative study. Women’s Health Issues, 21, S138–S144.

    Article  PubMed  Google Scholar 

  44. Tannenbaum, C., Greaves, L., & Graham, I. D. (2016). Why sex and gender matter in implementation research. BMC Medical Research Methodology, 16, 145.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Lukas, C. V., Holmes, S. K., Cohen, A. B., et al. (2007). Transformational change in health care systems: an organizational model. Health Care Management Review, 32, 309–320.

    Article  PubMed  Google Scholar 

  46. Mold, J. W., & Peterson, K. A. (2005). Primary care practice-based research networks: working at the interface between research and quality improvement. Annals of Family Medicine, 3, S12–S20.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Jeffs, L. P., Lo, J., Beswick, S., & Campbell, H. (2013). Implementing an organization-wide quality improvement initiative: insights from project leads, managers, and frontline nurses. Nursing Administration Quarterly, 37, 222–230.

    Article  PubMed  Google Scholar 

  48. Johnson Faherty, L., Mate, K. S., & Moses, J. M. (2016). Leveraging trainees to improve quality and safety at the point of care: three models for engagement. Academic Medicine, 91, 503–509.

    Article  PubMed  Google Scholar 

  49. Spears, W., Tsoh, J. Y., Potter, M. B., et al. (2014). Use of community engagement strategies to increase research participation in practice-based research networks (PBRNs). Journal of American Board of Family Medicine, 27, 763–771.

    Article  Google Scholar 

  50. Irvine, A. (2011). Duration, dominance and depth in telephone and face-to-face interviews: a comparative exploration. Intl J Qual Methods., 10, 202–220.

    Article  Google Scholar 

  51. Brandrud, A. S., Schreiner, A., Hjortdahl, P., Helljesen, G. S., Nyen, B., & Nelson, E. C. (2011). Three success factors for continual improvement in healthcare: an analysis of the reports of improvement team members. BMJ Quality and Safety, 20, 251–259.

    Article  PubMed  Google Scholar 

  52. Kaplan, H. C., Brady, P. W., Dritz, M. C., et al. (2010). The influence of context on quality improvement success in health care: a systematic review of the literature. The Milbank Quarterly, 88, 500–559.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Franx, G., Dixon, L., Wensing, M., & Pincus, H. (2013). Implementation strategies for collaborative primary care-mental health models. Current Opinion in Psychiatry, 26, 502–510.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Susan Frayne for her WH-PBRN leadership and Ms. Diane Carney for her management of the WH-PBRN; Ms. Selene Mak for data analysis support; Dr. Amy Cohen for editorial input; and Ms. Kristina Oishi and Ms. Anneka Oishi for logistics support. We would also like to thank Ms. Angela Cohen for her overall management of the Women’s Health CREATE, and our Executive Steering Committee members and Women Veteran Council members for their valuable contributions to this project.

Author information

Authors and Affiliations

  1. VA HSR&D Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, 11301 Wilshire Blvd., Building 206, Los Angeles, CA, 90073, USA

    Alison B. Hamilton PhD, MPH, Julian Brunner MPH, Ismelda Canelo MPA & Elizabeth M. Yano PhD, MSPH

  2. Department of Psychiatry and Biobehavioral Sciences, UCLA Geffen School of Medicine, Los Angeles, CA, 90095, USA

    Alison B. Hamilton PhD, MPH

  3. Department of Health Policy & Management, UCLA Fielding School of Public Health, 650 Charles E. Young Drive South, Los Angeles, CA, 90095-1772, USA

    Julian Brunner MPH, Cindy Cain PhD, Emmeline Chuang PhD, Tana M. Luger PhD, MPH & Elizabeth M. Yano PhD, MSPH

  4. Pitzer College Psychology Field Group, 1050 N Mills Ave, Claremont, CA, 91711, USA

    Tana M. Luger PhD, MPH

  5. RAND Health, 1776 Main Street, Santa Monica, CA, 90401-3208, USA

    Lisa Rubenstein MD, MSPH

  6. Department of Medicine, UCLA Geffen School of Medicine, 10833 Le Conte Avenue, Los Angeles, CA, 90095, USA

    Lisa Rubenstein MD, MSPH

Authors
  1. Alison B. Hamilton PhD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julian Brunner MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cindy Cain PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emmeline Chuang PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tana M. Luger PhD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ismelda Canelo MPA
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lisa Rubenstein MD, MSPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Elizabeth M. Yano PhD, MSPH
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alison B. Hamilton PhD, MPH.

Ethics declarations

Study funding sources

Support for this research was provided by VA HSR&D Service, Veterans Health Administration through the CREATE initiative (CRE 12-026) and the VA HSR&D Center for the Study of Healthcare Innovation, Implementation, and Policy (CIN 13-417). Development and implementation of the EBQI intervention is being supported through annual Memoranda of Understanding with funding from VA Women’s Health Services (WHS) in the Office of Patient Care Services, Veterans Health Administration. Dr. Yano’s effort was funded by a VA HSR&D Senior Research Career Scientist Award (Project #RCS 05-195).

Conflicts of interest

The authors declare that they have no conflicts of interest.

Reporting and control of data

The findings reported have not been previously published and the manuscript is not being simultaneously submitted elsewhere. The data has not been previously reported. The authors have full control of all primary data and agree to allow the journal to review their data if requested. The views expressed within this study are solely those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or the U.S. Government.

Ethics approval and consent to participate

This study protocol has been approved by the VA Greater Los Angeles Healthcare System IRB-B Subcommittee (IRB Chair, Paul Shekelle, MD, PhD) (approval number 2013-040589). Individual study components, including key stakeholder interviews, were reviewed and approved by IRB-B as serial project modifications to an umbrella IRB application under the study title. For each study component, we obtained IRB approval for waivers of documentation of consent for all data collections. Consent was provided verbally and recorded in the case of interviews. The VA Central IRB (Washington DC) also determined that the participating VA medical centers (n = 12) were not engaged in research and that the intervention (or implementation strategy, evidence-based quality improvement [EBQI]) was QI. Documentation of IRB approval (and in VA settings, Research and Development Committee approval as well) and Central IRB determination of non-research for the implementation strategy are available on request. The UCLA Institutional Review Board also reviewed and approved the key stakeholder interview component. No individual person’s data in any form are or will be included in this or subsequent manuscripts. Only aggregated data will be reported. For key stakeholder data, non-attributed or identifiable quotes may be reported in addition to thematic summaries.

Statement on welfare of animals

No animals are or will be involved in this study protocol. Similarly, use of animal or human data or tissue is not applicable to this study protocol.

Helsinki statement

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Additional information

Implications

Research: Researchers interested in engaging organizational stakeholders in implementation and improvement efforts could consider tailoring data reports to subpopulations of interest and using that data to educate, raise awareness, and promote buy-in. Researchers could also make efforts to align new initiatives with institutional norms and existing practices.

Practice: Practitioners interested in engaging in implementation and improvement efforts could, with institutional support, (1) seek out local researchers who are conducting studies on topics of interest to learn about opportunities for involvement, (2) seek out inter-disciplinary improvement initiatives to become involved in them or use them as a model for new initiatives, or (3) identify relevant training opportunities that might create linkages to individuals who are conducting interesting projects.

Policy: Policymakers interested in engaging in implementation and improvement efforts could identify ways to obtain data—analyzed and available in approachable ways—that would support decision-making around care targets that may warrant policy changes or innovations, and they could also partner with researchers to guide the design of studies that would generate policy-relevant results.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamilton, A.B., Brunner, J., Cain, C. et al. Engaging multilevel stakeholders in an implementation trial of evidence-based quality improvement in VA women’s health primary care. Behav. Med. Pract. Policy Res. 7, 478–485 (2017). https://doi.org/10.1007/s13142-017-0501-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13142-017-0501-5

Keywords

Search

Navigation