Abstract
The evaluation of public diplomacy programs presents complicated challenges. Discernment of impact is complicated by statistical and practical issues: the nature of individualized personal experiences; the large number of factors that can influence an individual’s response to any experience; the long time horizon required for impact to develop; the influence of politics on defining desired outcomes; and a longrunning debate within the discipline over the proper objectives of exchange programs (mutual understanding for its own sake or the pursuit of foreign policy agendas). Researcher asked current and former participants in the Hubert H. Humphrey Exchange Program at Syracuse University for opinions on the outcomes, benefits, and attributes they expect of exchange programs, and used Q methodology, a scientific method for the study of subjectivity, to discern and describe differing perspectives. Results obtained revealed distinct differences in the opinion patterns of different groups of participants, including identifying participants who valued more agenda- and policy-driven objectives. Demographic information obtained was insufficient to identify the drivers of those groups and additional research, including expansion of the respondent pool and analysis of individual participants, is needed to refine the precise drivers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baltrinic, E.R., M. Jencius, and S.R. Brown. 2017. Excellent Teaching as Viewed by International Teaching Elites. Operant Subjectivity 38 (3/4): 16–28.
Banasick, S. 2019. Ken-Q Analysis (Version 1.0.6) [Software]. https://shawnbanasick.github.io/ken-q-analysis/https://doi.org/10.5281/zenodo.1300201. Accessed Jan 2020.
Broderick, F., T. Deignan, and H. Combes. 2017. An Exploratory Study into the Traumatic Impact of Advanced Cancer among Patients and Partners. Operant Subjectivity 39 (3/4): 1–24.
Brown, S.R. 1980. Political subjectivity: Applications of Q Methodology in Political Science. New Haven, CT, USA: Yale University Press.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2000. Evaluation of the Hubert H. Humphrey Fellowship Program, Executive Summary. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2001a. Congress-Bundestag Youth Exchange Program (CBYXX), Executive Summary Briefing. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2001b. Evaluation Summary: Junior Faculty Development Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2002a. Outcome Assessment of the U.S. Fulbright Scholarship Program Executive Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2002b. Outcome Assessment of International Professional Exchange Programs in Philadelphia, Pennsylvania Executive Summary. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2003a. Outcome Assessment of the Institute for Representative Government (IRG) Program, Executive Summary. Washington, D.C. Retrieved between December 2020 and January 2021 from https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2003b. Assessment of the Future Leaders Exchange (FLEX) Program. Washington, D.C. Retrieved between December 2020 and January 2021 from https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2003c. FREEDOM Support Act Undergraduate Program, One-Page Summary. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2003d. Evaluation of the English Language Fellows Program, Executive Summary. Washington, D.C. Retrieved between December 2020 and January 2021 from https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2004. Evaluation of the FREEDOM Support Act Educational Partnerships Program, Executive Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2005. Outcome Assessment of the Visiting Fulbright Student Program Executive Summary. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2006. International Visitor Leadership Program Outcome Assessment. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2009. Evaluation of the Youth Exchange and Study Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2012. Evaluation of ECA’s Professional Journalism and Media Exchange Programs: International Visitor Leadership Program; Edward R. Murrow Program For Journalists; Citizen Exchange Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2015a. Evaluation of ECA’s English Language Programs: E-Teacher Scholarship Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2015b. Evaluation of ECA’s English Language Programs: English Language Specialist Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2016. English Access Microscholarship Program Evaluation Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2017a. African Women’s Entrepreneurship Program (AWEP-IVLP) Evaluation Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2017b. Evaluation of the Fulbright Foreign Student Program: Impact on STEM Participants. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2018. Hubert H. Humphrey Fellowship Program Evaluation Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2019. Alumni Engagement Innovation Fund Evaluation Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2019. Evaluation of Twelve Projects Supported by the U.S. Department of State Ambassadors Fund for Cultural Preservation. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2020a. Evaluation Report: International Writing Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2020b. Monitoring Data for ECA Framework Indicator Book, Version 2. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2020c. Evaluation of the Mandela Washington Fellowship 2014–2018. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2020d. Evaluation Report: Mandela Washington Fellowship of Young African Leaders. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2020e. Professional Fellows Program FY2012-FY1017 Evaluation Report. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Bureau of Educational and Cultural Affairs, U.S. Department of State. 2020f. Evaluation Report: Sports Visitor Program. Washington, D.C. https://eca.state.gov/impact/evaluation-eca/evaluation-initiative/completed-evaluations. Accessed between Dec 2020 and Jan 2021.
Cull, Nicholas J. 2019. Public Diplomacy: Foundations for Global Engagement in the Digital Age. Medfor, MA: Polity Press.
Cull, Nicholas J. 2008. Public Diplomacy: Taxonomies and Histories. Annals of the American Academy of Political and Social Science 616: 31–54.
DeVore-Wedding, B., J. Thomas, and D. Montgomery. 2018. Determining Optimal Professional Development Formats: A Q Methodology Study of Science Teachers’ Preferences. Operant Subjectivity 40: 1–20.
Hart, J. 2013. Empire of Ideas: The Origins of Public Diplomacy and the Transformation of U.S. Foreign Policy. New York: Oxford University Press.
Hedges, C.D. 2014. The Gender Factor of Survivor: A Q Method Approach. Operant Subjectivity 37 (1/2): 2–22.
Kinsey, D.F. 1991. Selecting a Winning Campaign Slogan. Operant Subjectivity 15 (1): 1–10.
Kinsey, D.F., and R.W. Taylor. 1982. Some Meanings of Political Cartoons. Operant Subjectivity 5 (3): 107–114.
McKeown, B., and Thomas, D.B. 2013. Q Methodology. Sage University Paper Series on Quantitative Applications in the Social Sciences, Vol. 07(66), 2nd ed.
Pahlavi, P.C. 2007. Evaluating Public Diplomacy Programmes. Hague Journal of Diplomacy 2: 255–281.
Pamment, J. 2014. Articulating Influence: Toward a Research Agenda for Interpreting the Evaluation of Soft Power, Public Diplomacy and Nation Brands. Public Relations Review 40: 50–59.
Pike, Steven L., and D.F. Kinsey. 2021. Diplomatic Identity and Communication: Using Q Methodology to Assess Subjective Perceptions of Diplomatic Practitioners. Place Branding and Public Diplomacy. https://doi.org/10.1057/s41254-021-00226-2.
Rhoads, J.C. 2017a. Foreword to the Special Issue: Q Methodology and the Single Case. Operant Subjectivity, 39(1–2): 1–100.
Rhoads, J.C. 2017b. Investigating Political Types, Part I: A Study of “Phil”, the Political Agitator. Operant Subjectivity 39 (1–2): 58–69.
Rhoads, J.C. 2017c. Investigating Political Types, Part II: A Study of “Cole”, the Political Administrator. Operant Subjectivity 39 (1–2): 70–81.
Rhoads, J.C., and D.F. Kinsey. 2008. Democratic Campaign: Perceptions of the Obama-Clinton Nomination Struggle. Operant Subjectivity 31 (1): 100–123.
Rhoads, J.C., D.B. Thomas, and B.F. McKeown. 2017. Rationality vs. Rationale Among Trump Voters in 2016: What Were They Thinking? Operant Subjectivity 39 (3/4): 60–80.
Robinson, T., C. Callahan, and K. Evans. 2014. Why Do We Keep Going Back? A Q Method Analysis of Our Attraction to Horror Movies. Operant Subjectivity 37 (1/2): 41–57.
Scott-Smith, Giles. 2008. Some Thoughts on the Relevance of Exchange Programs within International Relations Theory. ANNALS, AAPSS, 616.
Sevin, E. 2017. A Multilayered Approach to Public Diplomacy Evaluation: Pathways of Connection. Politics & Policy (Statesboro, Ga.) 45 (5): 879–901.
Simpson, N.P., and R.C. Hill. 2020. A Theoretically Inspired Q Methodological Approach for the Evaluation of Public Participation in Environmental Impact Assessment. Operant Subjectivity 42: 1–32.
Snow, Nancy. 2010. Propaganda Inc: Selling America’s Culture to the World. New York: Seven Stories Press.
Sommerfeldt, E.J., and Buhmann, A. 2019. The status quo of evaluation in public diplomacy: insights from the U.S. State Department. Journal of Communication Management 23 (3): 198–212.
Stephenson, W. 1953. The Study of Behavior: Q Technique and Its Methodology. Chicago, IL, USA: University of Chicago Press.
Stephenson, W. 1978. Concourse Theory of Communication. Communication 3: 21–40.
Stephenson, W. 1980. Consciring: A general theory for subjective communicability. In Communication yearbook 4, ed. D. Nimmo, 7–36. New Brunswick, NJ: Transaction.
VQMethod. (n.d.). VQMethod.com. http://www.vqmethod.com. Accessed Jan 2020.
Zenor, J. 2014. Where are Those Good Ol’ Fashioned Values? Reception Analysis of the Offensive Humor on Family Guy. Operant Subjectivity 37 (1/2): 23–40.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pike, S.L. Using Q methodology to augment evaluation of public diplomacy programs. Place Brand Public Dipl 18, 240–253 (2022). https://doi.org/10.1057/s41254-021-00229-z
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1057/s41254-021-00229-z