Abstract
Background
Patient activation is associated with better outcomes in chronic conditions.
Objective
We evaluated the psychometric properties of the 12-item Altarum Consumer Engagement™ Measure (ACE-12) in patients with prediabetes.
Participants
ACE-12 was administered to patients in the Prediabetes Informed Decisions and Education Study.
Main Measures
We conducted an exploratory factor analysis followed by confirmatory factor analytic models. We evaluated item response categories using item characteristic curves. Construct validity was assessed by examining correlations of the ACE-12 scales with education, depressive symptoms, self-rated health, hemoglobin A1c, body mass index, and weight loss.
Key Results
Participants (n = 515) had a median age of 58; 56% were female; 17% Hispanic; 54% were non-White. The scree plot and Tucker and Lewis reliability coefficient (0.95) suggested three factors similar to the original scales. One item loaded on the navigation rather than the informed choice scale. Ordinal alpha coefficients for the original scales were commitment (0.75); informed choice (0.71); and navigation (0.54). ICCs indicated that one or more of the response categories for 5 of the 12 items were never most likely to be selected.
Patients with lower education were less activated on the commitment (r = − 0.124, p = 0.004), choice (r = − 0.085, p = 0.009), and overall score (r = − 0.042, p = 0.011). Patients with depressive symptoms had lower commitment (r = − 0.313, p ≤ 0.001) and overall scores (r = − 0.172, p = 0.012). Patients with poorer health scored lower on the Commitment (r = − 0.308, p ≤ 0.001), Navigation (r = − 0.137, p ≤ 0.001), and overall score (r = − 0.279, p ≤ 0.001).
Conclusion
The analyses provide some support for the psychometric properties of the ACE-12 in prediabetic patients. Future research evaluating this tool among patients with other chronic conditions are needed to determine whether Q1 (I spend a lot of time learning about health) should remain in the informed choice or be included in the navigation scale. Additional items may be needed to yield acceptable reliability for the navigation scale.
Similar content being viewed by others
BACKGROUND
The Centers for Disease Control and Prevention estimates that more than 1 in 3 American adults have prediabetes and are at risk of progression to type 2 diabetes and it is deleterious long-term complications.1 The Diabetes Prevention Program (DPP) randomized controlled trial showed that the intensive lifestyle change reduced the incidence of progression to type 2 diabetes by 58% and metformin decreased the risk by 31%.2 Moreover, the follow-up DPP Outcomes Study demonstrated continuing positive health outcomes at 10- and 15-year follow-ups.3, 4 Despite this strong evidence supporting the importance of diabetes prevention, widespread translation of the DPP intervention into real-world settings continues to present challenges.
Behavioral interventions such as the DPP require engaged and activated patients to achieve optimal outcomes. Patient activation is patient self-efficacy for chronic care self-management and is an indicator of readiness for engagement in care.5 Patients who are younger,6, 7 more educated,6, 8 have higher income,6, 8 and better self-rated health tend to be more activated.6,7,8 Increases in patient activation and engagement have been associated with better self-care,7 disease-specific self-management,9,10,11,12 and health outcomes.13, 14
The Altarum Consumer Engagement (ACE) Measure™ is a patient engagement tool that includes questions about confidence in maintaining a role in one’s health, patterns of seeking information about health and healthcare, and asking about and participating in treatment decisions.15 The ACE was developed to identify factors that encourage adoption of patient involvement in their own healthcare and to evaluate programs designed to facilitate patient decision-making. This measure of patient engagement is unique in that it addresses dimensions of engagement that patients need to utilize information sources and tools. The ACE was administered in an online survey of 2079 adults in the general population. Based on principal components and Rasch analyses, the developers identified a 6-item Commitment scale, 5-item Informed Choice scale, 5-item Navigation scale, a 5-item Ownership scale, and a total activation scale.15 Spearman correlations of these scales with the 13-item Patient Activation Measure® ranged from 0.29 to 0.65.
The ACE was administered to a sample of adults with type II diabetes,16 but its psychometric properties in patients with prediabetes have not been previously published. We used data from the Prediabetes Informed Decisions and Education (PRIDE) study to evaluate the psychometric properties of the current version of the ACE that consists of 12 of the 21 original survey items (ACE-12, see Table 1) at baseline (https://altarum.org/sites/default/files/uploaded-related-files/ACE%20ProductSheet_091615f.pdf).
METHODS
Study Design
We analyzed data from the intervention arm of the PRIDE study, a cluster-randomized trial of a shared decision-making (SDM) intervention to help patients select an evidence-based diabetes prevention strategy (i.e., intense lifestyle change, metformin, or both). Patients in the intervention arm also had the option to choose no prevention strategy. The study was approved by the Institutional Review Board at the University of California, Los Angeles (IRB#15-000310).
Setting and Participants
The PRIDE study was conducted from 2015 to 2018 at UCLA Health.17 Twenty primary care clinics in the greater Los Angeles area were randomized to SDM intervention (10 clinics) or usual care (10 clinics). We used electronic medical record data to identify eligible patients. Inclusion criteria were age 18–74, overweight (body mass index [BMI] ≥ 24 kg/m2 or ≥ 22 kg/m2 if Asian) and a diagnosis of prediabetes (HbA1c 5.7–6.4% within the prior 3-months). Patients with diabetes (i.e., any HbA1c > 6.4%, ICD 250.xx billing code, oral antiglycemic medications and/or insulin), chronic kidney disease (i.e., estimated glomerular filtration rate ≤ 45 ml/min/1.73m2), active eating disorder(s), and women who had polycystic ovary syndrome or were pregnant or planning to get pregnant in the next year were excluded.
After eligible patients were identified, the primary care providers (PCPs) were notified and a formal standardized invitation letter signed by the PCP was sent to the patient via mail. A research assistant then invited patients in the intervention arm to an SDM visit with a clinical pharmacist. PCPs were also invited to refer patients with prediabetes to the study.
Intervention
The SDM intervention was delivered by clinical pharmacists embedded in each intervention clinic, who work closely with the clinic PCPs. The pharmacists used an online decision aid called “Prediabetes: Which Treatment Should I Use to Prevent Type 2 Diabetes?” to facilitate the SDM sessions. The decision aid was created by Healthwise, a national provider of health information and patient decision support tools for over 35 years. The decision aid provides information about prediabetes, compares intensive lifestyle changes and metformin as two evidence-based options for diabetes prevention, and summarizes the relative risk reduction as well as potential side effects of each option.17
Measures
We administered the ACE-12 items at baseline to participants prior to the SDM session and again at 4- and 12-month follow-up interviews. The ACE-12 uses a 5-point response scale: strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, and strongly agree, and higher scores represent greater patient engagement. Participants also completed the Patient Health Questionnaire (PHQ-9) at baseline and 4- and 12-month follow-up intervals. Other patient characteristics measured at baseline were age, race, ethnicity, income, education, body mass index (BMI), hemoglobin A1c, and comorbidities. Patient outcomes for the PRIDE study are reported in a separate manuscript.17, 18
Analysis
We conducted an exploratory factor analysis of the ACE-12 to explore whether any items loaded on different factors than those identified in the original developmental work. We identified the number of factors using the Tucker-Lewis reliability coefficient and the scree plot of eigenvalues with squared multiple correlations as communality estimates. Good model fit was evaluated by the comparative fit index (CFI) ≥ 0.95 and root mean square error of approximation (RMSEA) ≤ 0.06.19 We then fit confirmatory factor analytic models for the original ACE-12 scales and the variants on these scales suggested by the exploratory factor analysis. To minimize problems with overfitting in the sample, we also fit the confirmatory factor analytic models on a national representative mail and web survey of over 4000 consumers: https://altarum.org/publications/right-place-right-time. Good model fit was evaluated by the comparative fit index (CFI) ≥ 0.95 and root mean square error of approximation (RMSEA) ≤ 0.06.19
Each response category for items in a survey should have the highest likelihood of being selected at some area of the distribution and the categories should be ordered monotonically with respect to the underlying concept being measured. We evaluated the ACE-12 response options using item characteristic curves (ICCs) based on the graded response model.20
We then evaluated construct validity by estimating associations of the ACE-12 scales with education level, depression symptoms, self-reported health, hemoglobin A1c, BMI at baseline, and weight loss over 12 months. We hypothesized that those who chose an intervention versus those who chose not to adopt an evidence-based prevention strategy after SDM, those who have more education (greater than a high school education) and better self-reported health (‘very good” or “excellent”) would have higher activation scores. We also hypothesized that more activated patients would be more likely to complete the DPP and lose 5–7% of their baseline weight. In contrast, based on evidence from prior literature we hypothesized that patients with more depressive symptoms (PHQ-9 score ≥ 5),21 those with higher A1c16, and those who are obese (BMI ≥ 30.0) would be less activated.15, 16, 22 We report product-moment correlations and associated effect sizes (Cohen’s d, where d = (2*r)/SQRT(1 − r2). Cohen’s conventions for magnitude of correlations were used: > 0.100 to < 0.243 = small; 0.243 to < 0.371 = medium; and ≥ 0.371 = large.23
Confirmatory factor analysis models were estimated using Mplus version 7.24 All other analyses were conducted with SAS 9.3 (SAS Institute, Inc., Cary, NC).
RESULTS
Participant Characteristics
Sample characteristics are given in Table 2. The sample (n = 515) had a mean age of 56 (range = 45–67); 56% were female; 17% were Hispanic, 83% were non-Hispanic. The majority of participants were White (46%), followed by Asian/Pacific Islander (19%), African American (15%), Native American (1%), and Other (19%). The mean A1c was 5.95 (SD = 0.2) and mean BMI was 30.31 (SD = 5.16) at baseline.
Factor Analyses
The scree plot (Fig. 1) and Tucker and Lewis reliability coefficient (0.95) suggested three underlying factors. The rotated three-factor solution provided confirmation of the ACE-12 scales. However, we found that item #1 (I spend a lot of time learning about health) loaded on the navigation factor rather than on the informed choice factor.
Fit of the confirmatory factor analysis models are summarized in Table 3. Single-factor models for the original scales and the variants of the informed choice and navigation scales fit the data well according to the comparative fit index (i.e., ≥ 0.95) but RMSEA values for the commitment and informed choice scales exceeded the threshold (≤ 0.06) for good fit. The three-factor model for the revised version of the scales fit the data better than did the original scales. Ordinal alpha25 was 0.75 for the commitment scale and alphas were similar for the original and the variants, respectively, of informed choice (0.71 and 0.78), and navigation (0.54 and 0.62).
Item Characteristic Curves
ICCs are shown in Figure 2 and grouped by ACE-12 scales. Seven of the items had response categories performing as expected (i.e., each category had the highest likelihood of being selected at some area of the distribution and was ordered monotonically with respect to the underlying concept). There were issues with the other five items. The neither agree nor disagree option was never most likely to be chosen for Q2 (commitment, Fig. 2a) and Q6 (informed choice, Fig. 2b). The strongly disagree option was never most likely to be selected for Q1 (informed choice, Fig. 2b) and Q5 (navigation, Fig. 2c). The somewhat disagree and strongly agree options were never most likely to be chosen for Q3 (navigation, Fig. 2c).
Construct Validity
Correlations of the ACE-12 scales are presented in Table 4. The pattern of correlations was similar for the original scales and the variants of the informed choice and navigation scales. Hence, we summarize only the correlations for the original scales below. Patients with a high school education or lower at baseline scored as less activated on the commitment scale (r = − 0.124, d = − 0.250; p = 0.004), choice scale (r = − 0.085, d = − 0.171, p = 0.009), and total activation score (r = − 0.042, d = − 0.084, p = 0.011) at baseline. Education level at baseline was not associated significantly with the learning about health scale. Whether an individual selected an intervention and A1c at baseline were not significantly correlated with activation scores. Depressive symptoms correlated significantly with the commitment scale (r = − 0.313, d = − 0.659, p < 0.001) and total activation score (r = − 0.172, d = − 0.349, p = 0.012). Specifically, those who scored 5 or higher on the PHQ-5 reported scored less on the commitment scale and the overall ACE-12 score. Depressive symptoms had the largest magnitude of association with the commitment scale (r = − 0.313, d = − 0.659, p ≤ 0.001). Lastly, patients who self-reported their health as less than “very good” scored lower on the commitment scale (r = − 0.308, d = − 0.647, p < 0.001), navigation scale (r = − 0.137, d = − 0.277, p ≤ 0.001), and total activation score (r = − 0.279, d = − 0.581, p ≤ 0.001). Except for the choice scale, all the original ACE scales were significantly correlated with self-reported health. Those who were obese (BMI ≥ 30.0) scored less on the commitment scale (r = − 0.177, d = − 0.360, p = 0.001), and total activation scale (r = − 0.136, d = − 0.275, p = 0.016). Weight change over 12 months was not associated with greater activation scores at baseline.
DISCUSSION
The results of this study provide some support for the original ACE-12 scales derived by the Duke et al.15 This is important given the need to better measure and understand patient activation among patients with chronic conditions for which behavior change and/or chronic medication use are recommended. Because of the challenges in recruiting and retaining patients into behavior change programs26 and supporting patients in adherence to metformin for diabetes prevention, measuring activation at baseline can be useful in identifying patients who may benefit from extra support.
In terms of hypothesized correlations between patient activation and other variables, ACE-12 scores in our study were negatively associated with education, depressive symptoms, and obesity at baseline. Similarly, our analysis showed that higher ACE scores correlated with better self-reported health at baseline. Less patient activation was observed for those with more depressive symptoms. These findings are generally consistent with a large body of literature showing that patients with diabetes and depression often have difficulty engaging in lifestyle modification,27 adherence to medications,28, 29 and report worse quality of life30 and health outcomes.31 Likewise, consistent with existing literature,15, 22 obesity was associated with less patient activation in our study.
Interestingly, patients with less education reported higher activation at baseline. This finding is contrary to prior research showing less activation among those who are less educated.32 One possible explanation is that study participants with less education may have had less baseline understanding of prediabetes and were highly motivated to enroll in the study and learn about their condition and prevention strategies to avoid diabetes. Similarly, patients with more education who were highly activated may have already raised this issue with their PCP and/or sought information from other sources and may have been less interested in participating in SDM for diabetes prevention. Lastly, we found that better self-reported health at baseline had the largest association with patient activation.
Our study has several limitations. First, the sample was comprised of overweight/obese patients with prediabetes from one large medical system, limiting the generalizability of the findings. In addition, construct validity analyses relied primarily on data gathered at baseline. Future studies may want to analyze construct validity by exploring longitudinal relationships among the variables. Hence, the direction of effects cannot be known from the associations reported.
Given the relationship between patient activation and engagement and improved health outcomes, our study provides support for the ACE-12, a brief patient activation tool that can be easily used and is accessible for research, quality improvement, or clinical use without licensing fees. Future studies evaluating this tool among patients with multiple chronic conditions will be needed to determine whether Q1 (I spend a lot of time learning about health) should remain in the informed choice or be included in the navigation scale. In addition, the reliability of navigation scale was suboptimal (0.54) so it may be important to include additional items to achieve adequate reliability for this construct.
References
Center for Disease Control. The Surprising Truth About Prediabetes. Available at: https://www.cdc.gov/features/diabetesprevention/index.html. Accessed December 18, 2018.
Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. The New England journal of medicine. 2002;346(6):393–403.
Long-term effects of lifestyle intervention or metformin on diabetes development and microvascular complications over 15-year follow-up: the Diabetes Prevention Program Outcomes Study. The lancet Diabetes & endocrinology. 2015;3(11):866–875.
Knowler WC, Fowler SE, Hamman RF, et al. 10-year follow-up of diabetes incidence and weight loss in the Diabetes Prevention Program Outcomes Study. Lancet (London, England). 2009;374(9702):1677–1686.
Rodriguez HP, Poon BY, Wang E, Shortell SM. Linking Practice Adoption of Patient Engagement Strategies and Relational Coordination to Patient-Reported Outcomes in Accountable Care Organizations. Milbank Q. 2019.
Hibbard J, Cunningham P. How engaged are consumers in thier health and health care, and why does it matter? Res Briefs. 2008;8:1–9.
Zimbudzi E, Lo C, Ranasinha S, et al. The association between patient activation and self-care practices: A cross-sectional study of an Australian population with comorbid diabetes and chronic kidney disease. Health Expect. 2017;20(6):1375–1384.
Lubetkin EI, Lu WH, Gold MR. Levels and correlates of patient activation in health center settings: building strategies for improving health outcomes. J Health Care Poor Underserved. 2010;21(3):796–808.
Mosen DM, Schmittdiel J, Hibbard J, Sobel D, Remmers C, Bellows J. Is patient activation associated with outcomes of care for adults with chronic conditions? J Ambul Care Manage. 2007;30(1):21–29.
Hibbard JH, Mahoney ER, Stock R, Tusler M. Do increases in patient activation result in improved self-management behaviors? Health Serv Res. 2007;42(4):1443–1463.
Shively MJ, Gardetto NJ, Kodiath MF, et al. Effect of patient activation on self-management in patients with heart failure. J Cardiovasc Nurs. 2013;28(1):20–34.
Yadav UN, Hosseinzadeh H, Lloyd J, Harris MF. How health literacy and patient activation play their own unique role in self-management of chronic obstructive pulmonary disease (COPD)? Chronic respiratory disease. 2019;16:1479973118816418.
Mayberry R, Willock RJ, Boone L, Lopez P, Qin H, Nicewander D. A High Level of Patient Activation Is Observed But Unrelated to Glycemic Control Among Adults With Type 2 Diabetes. Diabetes Spectr. 2010;23(3):171–176.
Marshall R, Beach MC, Saha S, et al. Patient activation and improved outcomes in HIV-infected patients. J Gen Intern Med. 2013;28(5):668–674.
Duke CC, Lynch WD, Smith B, Winstanley J. Validity of a New Patient Engagement Measure: The Altarum Consumer Engagement (ACE) Measure. Patient. 2015;8(6):559–568.
Wardian J, Bersabe D, Duke C, Sauerwein TJ. Patient Commitment and Its Relationship to A1C. Clinical diabetes : a publication of the American Diabetes Association. 2018;36(4):295–304.
Moin T, Duru OK, Turk N, et al. Effectiveness of Shared Decision-making for Diabetes Prevention: 12-Month Results from the Prediabetes Informed Decision and Education (PRIDE) Trial. J Gen Intern Med. 2019;34(11):2652–2659.
Skrine Jeffers K, Castellon-Lopez Y, Grotts J, et al. Diabetes Prevention Program attendance is associated with improved patient activation: Results from the Prediabetes Informed Decisions and Education (PRIDE) study. Prev Med Rep. 2019;16:100961.
Hu L, Bentler, PM. Cutofff criteria for fit indicies in covariance analysis: conventional criteria versus new alternatives. Struc Equation Modeling. 1999(6 (1)):1–55.
Samejima F. The graded response model. In: Handbook of modern item response theor. New York, NY: Springer; 1996.
Magnezi R, Glasser S, Shalev H, Sheiber A, Reuveni H. Patient activation, depression and quality of life. Patient Educ Couns. 2014;94(3):432–437.
Greene J, Hibbard JH. Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes. J Gen Intern Med. 2012;27(5):520–526.
Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York, NY: Academic Press; 1988.
Muthén L, Muthén BO. Mplus user's guide. 7th ed. Los Angeles, CA: Muthén & Muthén; 2012.
Spritzer K, Hays, RD. A SAS macro (ordalpha) to compute ordinal Coefficient Alpha. https://drhays.dgsom.ucla.edu/files/view/docs/programs-utilities/SASOrdinalAlpha.pdf. Published 2015. Accessed October 2018.
Ely EK, Gruss SM, Luman ET, et al. A National Effort to Prevent Type 2 Diabetes: Participant-Level Evaluation of CDC's National Diabetes Prevention Program. Diabetes Care. 2017;40(10):1331–1341.
Swardfager W, Yang P, Herrmann N, et al. Depressive symptoms predict non-completion of a structured exercise intervention for people with Type 2 diabetes. Diabet Med. 2016;33(4):529–536.
Gonzalez JS, Peyrot M, McCarl LA, et al. Depression and diabetes treatment nonadherence: a meta-analysis. Diabetes Care. 2008;31(12):2398–2403.
Katz LL, Anderson BJ, McKay SV, et al. Correlates of Medication Adherence in the TODAY Cohort of Youth With Type 2 Diabetes. Diabetes Care. 2016;39(11):1956–1962.
Feng L, Yap KB, Ng TP. Depressive symptoms in older adults with chronic kidney disease: mortality, quality of life outcomes, and correlates. The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry. 2013;21(6):570–579.
Lin EHB, Katon W, Von Korff M, et al. Relationship of depression and diabetes self-care, medication adherence, and preventive care. Diabetes Care. 2004;27(9):2154–2160.
van Vugt HA, Boels AM, de Weerdt I, de Koning EJ, Rutten GE. Patient activation in individuals with type 2 diabetes mellitus: associated factors and the role of insulin. Patient preference and adherence. 2019;13:73–81.
Acknowledgments
We would like to thank Chris Duke, Ph.D. and Altarum for allowing us to use the Altarum Consumer Engagement (ACE) Measure.
Funding
This study was funded by the National Institute of Diabetes and Digestive and Kidney Diseases (R18DK105464). The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the views the National Institutes of Health (NIH). Dr. Castellon-Lopez received support from the University of California, Los Angeles (UCLA), Resource Centers for Minority Aging Research Center for Health Improvement of Minority Elderly (RCMAR/CHIME) under NIH/NIA Grant P30-AG021684, and from the UCLA Clinical and Translational Science Institute (CTSI) under NIH/NCATS Grant Number UL1TR001881. Dr. Skrine Jeffers’ research was supported by NIH National Center for Advancing Translational Science (NCATS) UCLA CTSI Grant Number TL1TR001883.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Prior Presentations Society of Teachers in Family Medicine National Conference 05/2019, UCLA-RCMAR/CHIME Regional Conference 06/2019, Family Medicine Multi-Campus Research Day at UCLA 06/2019.
Rights and permissions
About this article
Cite this article
Castellon-Lopez, Y., Skrine Jeffers, K., Duru, O.K. et al. Psychometric Properties of the Altarum Consumer Engagement (ACE) Measure of Activation in Patients with Prediabetes. J GEN INTERN MED 35, 3159–3165 (2020). https://doi.org/10.1007/s11606-020-05727-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-020-05727-z