Skip to main content
SpringerLink
Log in

Thin Descriptions: The Limits of Survey Research on the Meaning of Democracy

  • Article
  • Published:
Polity

Abstract

Survey researchers have produced a body of evidence suggesting that people’s understandings of democracy are surprisingly consistent worldwide. This article challenges that finding by comparing the results of a 2002 survey conducted in the Philippines with the results of my own 2001 fieldwork in one Philippine community where, using interpretive rather than survey-research tools, I also investigated how people understand democracy. The article identifies three generic methodological problems—compression, compartmentalization, and homogenization—that have led survey researchers in the Philippines and beyond to simplify meanings and falsely twin roughly equivalent words in different languages. The global consistency in meaning that survey researchers have discovered appears to be the product not of converging worldviews, but of specific procedures used to record, code, and interpret interview responses. Insofar as democracy-promotion initiatives are shaped by such surveys, their misleading quality is not only of methodological concern, but of immediate political relevance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Michaelle L. Browers, Democracy and Civil Society in Arab Political Thought: Transcultural Possibilities (Syracuse, NY: Syracuse University Press, 2006); Ellen Carnaghan, “The Difficulty of Measuring Support for Democracy in a Changing Society: Evidence from Russia,” Democratization 18 (June 2011): 682–706; Mikael Karlström, “Imagining Democracy: Political Culture and Democratisation in Buganda,” Africa 66 (October 1996): 485–505; Ahmed Khanani, “ ‘Two Hundred Percent Democracy?’ Islamism and Democracy in Morocco.” Ph.D. Thesis, Indiana University, 2014; Frederic Charles Schaffer, Democracy in Translation: Understanding Politics in an Unfamiliar Culture (Ithaca, NY: Cornell University Press, 1998), and “Ordinary Language Interviewing,” in Interpretation and Method: Empirical Research Methods and the Interpretive Turn, 2nd ed., ed. Dvora Yanow and Peregrine Schwartz-Shea (Armonk, NY: M. E. Sharp, 2014), 183–93.

  2. See, for example, Jose Abueva and Linda Luz Guerrero, “What Democracy Means to Filipinos,” Asian Barometer Working Paper 5 (Taipei: Asian Barometer Project Office, 2003); Robert B. Albritton and Thawilwadee Burkeel, “Developing Electoral Democracy in a Developing Nation: Thailand,” Asian Barometer Working Paper 17 (Taipei: Asian Barometer Project Office, 2004); Michael Bratton and Robert Mattes, “Support for Democracy in Africa: Intrinsic or Instrumental?” British Journal of Political Science 31 (July 2001): 447–74; Michael Bratton, Robert Mattes, and E. Gyimah-Boadi, Public Opinion, Democracy, and Market Reform in Africa (New York: Cambridge University Press, 2005); Roderic Ai Camp, “Democracy through Latin American Lenses: An Appraisal,” in Citizen Views of Democracy in Latin America, ed. Roderic Ai Camp (Pittsburgh: University of Pittsburgh Press, 2001), 3–23; Damarys Canache, “Citizens’ Conceptualizations of Democracy: Structural Complexity, Substantive Content, and Political Significance,” Comparative Political Studies 49 (September 2012): 1132–58; Yun-han Chu, Larry Diamond, Andrew J. Nathan, and Doh Chull Shin, eds., How East Asians View Democracy (New York: Columbia University Press, 2008); Russell J. Dalton, Doh C. Shin, and Willy Jou, “Understanding Democracy: Data from Unlikely Places,” Journal of Democracy 18 (October 2007): 142–56; Larry Diamond and Marc F. Plattner, eds., How People View Democracy (Baltimore: Johns Hopkins University Press, 2008); Dieter Fuchs and Edeltraud Roller, “Learned Democracy: Support of Democracy in Central and East Europe,” International Journal of Sociology 36 (Fall 2006): 70–96; Wai-man Lam and Hsin-chi Kuan, “Noises and Interruptions—The Road to Democracy,” Asian Barometer Working Paper 11 (Taipei: Asian Barometer Project Office, 2003); Richard R. Marcus, Kenneth Mease, and Dan Ottenmoeller, “Popular Definitions of Democracy from Uganda, Madagascar, and Florida, U.S.A,” Journal of Asian and African Studies 36 (January 2001): 113–32; Arthur H. Miller, Vicki L. Hesli, and William M. Reisinger, “Conceptions of Democracy among Mass and Elite in Post-Soviet Societies,” British Journal of Political Science 27 (April 1997): 157–90; Dan Ottemoeller, “Popular Perceptions of Democracy: Elections and Attitudes in Uganda,” Comparative Political Studies 31 (February 1998): 98–124; Tianjian Shi, “Is There an Asian Value? Popular Understanding of Democracy in Asia,” in China’s Reforms at 30: Challenges and Prospects, ed. Dali L. Yang and Litao Zhao (Hackensack, NJ: World Scientific, 2009), 167–94; Tianjian Shi and Jie Lu, “The Shadow of Confucianism,” Journal of Democracy 21 (October 2010): 123–30; Doh Chull Shin, “How East Asians Understand Democracy: From a Comparative Perspective,” paper presented at the Asian Barometer Conference on the State of Democratic Governance in Asia, Taipei, Taiwan, June 20–21, 2008; János Simon, “Popular Conceptions of Democracy in Postcommunist Europe,” in The Postcommunist Citizen, ed. Samuel H. Barnes and János Simon (Budapest: Erasmus Foundation and Institute for Political Science of the Hungarian Academy of Sciences, 1998), 79–116; Mark Tessler, Amaney Jamal, and Michael Robbins, “New Findings on Arabs and Democracy,” Journal of Democracy 23 (October 2012): 89–103.

  3. I find it useful to speak of demokrasya rather than “democracy” since we cannot assume a priori that Tagalog demokrasya—or Chinese minzhu or Luganda eddembe ery’obuntu or Wolof demokaraasi, and so on—mean exactly the same thing as English democracy (for elaboration, see Schaffer, Democracy in Translation, 140–46). By retaining the original Tagalog word, we are less likely to project onto this term meanings specific to “democracy.” The word demokrasya itself derives from the Spanish word democracia.

  4. A partial list: Abueva and Guerrero, “What Democracy Means”; Albritton and Bureekul, “Developing Electoral Democracy”; Bratton and Mattes, “Support for Democracy”; Canache, “Citizens’ Conceptualizations”; Dalton, Shin, and Jou, “Understanding Democracy”; Lam and Kuan, “Noises and Interruptions”; Miller, Hesli, and Reisinger, “Conceptions of Democracy”; Shi, “Is There an Asian Value”; Shi and Lu, “Shadow”; Shin, “How East Asians View Democracy.”

  5. Bratton, Mattes, and Gyimah-Boadi, Public Opinion; Chu et al., How East Asians View Democracy; Diamond and Plattner, How People View.

  6. Global Barometer, “Objectives,” at http://www.globalbarometer.net/objectives.htm (accessed February 12, 2014), can be accessed at http://webcache.googleusercontent.com/search?q=cache:CBCrJehfUboJ:www.globalbarometer.net/objectives.htm+&cd=8&hl=en&ct=clnk&gl=us.

  7. Dalton, Shin, and Jou, “Understanding Democracy,” 145, 154 n12.

  8. Not every iteration of the Global Barometer survey uses the open-ended question. Because this question has yielded consistent results across successive rounds of surveying, the item is no longer considered a high-enough priority to include in every round. Also, the Arab Barometer affiliate uses a closed-ended question.

  9. Dalton, Shin, and Jou, “Understanding Democracy,” 147.

  10. Ibid.

  11. Ibid., 154 n13.

  12. Yun-han Chu, Larry Diamond, Andrew J. Nathan, and Doh Chull Shin, “Comparative Perspectives on Democratic Legitimacy in East Asia” in their How East Asians View Democracy, 1–38.

  13. Michael Bratton, “Democratic Attitudes and Political Participation: An Exploratory Comparison across World Regions,” paper presented at the Congress of the International Political Science Association, Santiago, Chile, July 2009, 7.

  14. Linda Luz Guerrero and Rollin F. Tusalem, “Mass Public Perceptions of Democratization in the Philippines: Consolidation or Progress?” in Yun-han et al., How East Asians View Democracy, 66.

  15. Abueva and Guerrero, “What Democracy Means,” 4.

  16. We randomly selected 2 percent of the roughly 14,000 people listed on the voter registry for four sections of Commonwealth. This yielded a sample of 278 people. Two had died, and 107 had left this transient community after registering. Of the 171 people who still lived there, our response rate was 81 percent; hence the 139 interviews. Before this sampling, we conducted eleven pilot interviews in Commonwealth with people previously unknown to us, though we did not select them randomly from the voter registry. Statements from the shopkeeper, which we will encounter later in the article, are drawn from one of the pilot interviews. I do not include the pilot interviews in any statistical calculations.

  17. For a more in-depth description and explanation, see Schaffer, “Ordinary Language Interviewing.”

  18. Sixteen percent of the people we interviewed either chose not to answer our questions about demokrasya or stated that they did not know what the term means.

  19. Responses from an additional 304 interviewees were recorded as “missing,” and responses from another 15 were recorded as “none.”

  20. Dalton, Shin, and Jou, “Understanding Democracy,” 144–45.

  21. In other surveys, the problem of compression in the meaning-of-democracy question is even more acute. A survey conducted in eleven post-communist European countries limited answers to one sentence (Simon, “Popular Conceptions,” 85). Surveys conducted in three Latin American countries as well as the United States allowed respondents to supply only one word (Camp, “Democracy,” 17, 258). It is from these surveys that Dalton, Shin, and Jou draw their data for several European countries and the United States (Dalton, Shin, and Jou, “Understanding Democracy,” 154 n12).

  22. Abueva and Guerrero, “What Democracy Means,” 4.

  23. Two exceptions are worth noting: Simon theorizes about “linked responses” in post-communist Europe, while Shin examines how frequently people in East Asia hold “multidimensional” conceptions of democracy (Simon, “Popular Conceptions,” 97–98; Shin, “How East Asians View Democracy,” 16–20). Not coincidentally, the latter author questions the claim that liberal understandings of democracy predominate in East Asia. For a critique of compartmental thinking with regard to the interpretation of Afrobarometer survey results from South Africa, see Elke Zuern, “Democratization as Liberation: Competing Perspectives on Democracy,” Democratization 16 (June 2009): 585–603.

  24. John U. Wolff, A Dictionary of Cebuano Visayan, Volume 1 (Ithaca, NY: Southeast Asia Program, Department of Asian Studies, Cornell University, 1972), 261–62.

  25. John Kaufmann, Visayan-English Dictionary (Iloilo: La Editorial, 1935), 292; Edilberto N. Alegre and Doreen G. Fernandez, Writers and their Milieu: An Oral History of Second Generation Writers in English (Manila: De La Salle University Press, 1987), 198–99.

  26. Maria Angeles Guanzon-Lapeña and Robert Javier, Jr., “The Influence of an NGO on the Political Culture of a Community,” in Democracy & Citizenship in Filipino Political Culture, ed. Maria Serena I. Diokno (Quezon City: Third World Studies Center, 1997), 284–85; Carl Ralph Galvez Rubino, Ilocano Dictionary and Grammar (Honolulu: University of Hawai‘i Press, 2000), 642.

  27. Abdon M. Balde, Jr., “Bikol: Wika ng mga Maginoong Oragon,” paper presented at Ambagan-Wika: Kumperensiya sa Paglikom ng Salita Mula sa Iba’t ibang Wika sa Filipinas, University of the Philippines Diliman, March 5, 2009, 14–15.

  28. Sixty-seven percent of the Tagalog speakers who responded to the meaning-of-democracy question in the 2002 survey reportedly used the words kalayaan or malaya in their answers.

  29. J. L. Austin, Philosophical Papers, 2nd ed. (New York: Oxford University Press, 1970), 201.

  30. Ibid.

  31. Sevilla’s Tagalog spelling was calaya-an. Although the Tagalog edition I consulted dates to 1878, Sevilla began work on the translation in 1863, completed it in 1865, and was granted a license to print it by the Archbishop of Manila in 1867 after a review of its content. Mariano Sevilla, Flores de María ó Mariquit na Bulaclac (Manila: Establecimiento tipográfico de los Amigos del País, 1878), 3, 82; Moises B. Andrade and Edgar S. Yanga, Kalayaan: Its Birth and Growth Among the Secular Clergy in Bulacan (Bocaue, Bulacan: IBMA Print, 1998), 2.

  32. Sevilla, Flores, 82.

  33. Andrade and Yanga, Kalayaan, 3–4.

  34. Reynaldo Clemena Ileto, Pasyon and Revolution: Popular Movements in the Philippines, 1840–1910 (Quezon City: Ateneo de Manila University Press, 1979), 87, and “Critical Issues in ‘Understanding Philippine Revolutionary Mentality,’” Philippine Studies 30 (1982): 112–14.

  35. John N. Schumacher, Revolutionary Clergy: The Filipino Clergy and the Nationalist Movement, 1850–1903 (Quezon City: Ateneo de Manila University Press, 1981), 37.

  36. Andrade and Yanga, Kalayaan, 5; John N. Schumacher, The Propaganda Movement: 1880–1895 (Manila: Solidaridad, 1973), 94–114.

  37. Ileto, Pasyon and Revolution, 87.

  38. Vicente L. Rafael, “Translating Kalayaan,” in Thought the Harder, Heart the Keener: A Festschrift for Soledad S. Reyes, ed. Eduardo José E. Calasanz, Jonathan Chua, and Rofel G. Brion (Quezon City: Office of Research and Publications, Loyola Schools, Ateneo de Manila University, 2008), 112.

  39. William Henry Scott, “Filipino Class Structure in the 16th Century,” in Cracks in the Parchment Curtain and Other Essays in Philippine History (Quezon City: New Day, 1985), 99–111.

  40. Ibid., 100. Sixteenth-century Tagalog speakers on the island of Luzon, the largest island in the Philippine archipelago, differentiated ordinary timawa (people free to make contractual relations with maginoo) from aristocratic maharlika (military servants of maginoo) (Ibid., 104–106). Well before the late nineteenth century, however, the maharlika had disappeared as a social group; and Tagalog-Spanish dictionaries of the mid- and late-nineteenth century typically defined the word simply as a “free person.” By the time the nationalists were casting about for a suitable vocabulary, timawa and kamaharlikaan had become roughly synonymous and may have meant something like “not being indebted or bound to a particular patron or chief.”

  41. Isaiah Berlin, Four Essays on Liberty (New York: Oxford University Press, 1969), 122.

  42. As well, the 2002 survey was more proximate in time, and thus more comparable, to my own 2001 fieldwork.

  43. See, for instance, David Kelly and Anthony Reid, eds., Asian Freedoms: The Idea of Freedom in East and Southeast Asia (New York: Cambridge University Press, 1998); Robert H. Taylor, ed., The Idea of Freedom in Asia and Africa (Stanford, CA: Stanford University Press, 2002); Carnaghan, “Difficulty of Measuring,” 691; Anna Wierzbicka, Understanding Cultures through Their Key Words: English, Russian, Polish, German, and Japanese (New York: Oxford University Press, 1997), 125–55. Note, too, that not even the English words “freedom” and “liberty” mean the same thing, as shown by Hanna Fenichel Pitkin, “Are Freedom and Liberty Twins?” Political Theory 16 (November 1988): 523–52.

  44. Bratton, “Democratic Attitudes,” 7.

  45. Michael Bratton, “Anchoring the ‘D-Word’ in Africa,” Journal of Democracy 21 (October 2010): 107.

  46. Dalton, Shin, and Jou, “Understanding Democracy,” 147.

  47. Andreas Schedler and Rodolfo Sarsfield, “Democrats with Adjectives: Linking Direct and Indirect Measures of Democratic Support,” European Journal of Political Research 46 (August 2007): 637–59; Steven R. Brown, Political Subjectivity: Applications of Q Methodology in Political Science (New Haven: Yale University Press, 1980). See, too, the works I discuss in Note 23.

  48. Laura Stoker, “Is it Possible to do Quantitative Survey Research in an Interpretive Way?” Qualitative Methods 1 (Fall 2003): 13–16; Elizabeth Theiss-Morse, Amy Fried, John L. Sullivan, and Mary Dietz, “Mixing Methods: A Multistage Strategy for Studying Patriotism and Citizen Participation,” Political Analysis 3 (1991): 89–121.

  49. Bratton, “Anchoring the ‘D-Word,’” 109–12.

  50. Gary King, Christopher J. L. Murray, Joshua A. Salomon, and Ajay Tandon, “Enhancing the Validity of Cross-Cultural Comparability of Measurement in Survey Research,” American Political Science Review 98 (February 2004): 191, 192.

  51. Ibid., 194.

  52. Of course, no research method, not even ordinary language interviewing, can totally eliminate the problem of compression. Even when interviewees are given ample opportunities to speak, the researcher must still select and edit the raw interview transcripts for presentation, a procedure that is also in some sense compressive. Nevertheless, the compression introduced by survey-research instruments is distinctively problematic. The data collected by means of these instruments are from the get-go reductive. For a more general discussion of compression in ethnographic work, see Timothy Pachirat, Every Twelve Seconds: Industrialized Slaughter and the Politics of Sight (New Haven: Yale University Press, 2011), 285 n20.

  53. Gilbert Ryle, Collected Papers Volume II, 1929–1968 (London: Hutchinson, 1971), 465–96; Clifford Geertz, “Thick Description: Toward an Interpretive Theory of Culture,” in The Interpretation of Cultures (New York: Basic, 1973), 9–12.

  54. Ryle, Collected Papers, 482.

  55. Ludwig Wittgenstein, Philosophical Investigations, 3rd ed. (New York: Palgrave Macmillan, 1972), Paragraph 593.

  56. Ibid., Paragraph 66.

  57. Ibid.

  58. See Browers, Democracy and Civil Society; Carnaghan, “Difficulty in Measuring;” Karlström, “Imagining Democracy;” Khanani, “Islamism and Democracy in Morocco;” Schaffer, Democracy in Translation and “Ordinary Language Interviewing.”

  59. Tony Smith, A Pact with the Devil: Washington’s Bid for World Supremacy and the Betrayal of the American Promise (New York: Routledge, 2007), 132–62.

  60. On earlier uses of comparative research for this end, see Susanne Hoeber Rudolph, “The Imperialism of Categories: Situating Knowledge in a Globalizing World,” Perspectives on Politics 3 (March 2005): 5–14.

Author information

Authors and Affiliations

  1. University of Massachusetts, Amherst

    Frederic Charles Schaffer

Authors
  1. Frederic Charles Schaffer
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

The author gratefully acknowledges the Fulbright Scholars Program for supporting his fieldwork in the Philippines. He is also indebted to Robert Adcock, Aries Arugay, Ivan Ascher, Emily Beaulieu, Patrick Jackson, Andreas Schedler, Peregrine Schwartz-Shea, Dvora Yanow, and the anonymous Polity reviewers for their insightful comments; to Nancy Kimuell-Gabriel for generously sharing hard-to-find materials; and to Ramon V. Calleja Jr., Linda Luz Guerrero, and Jeanette Ureta of the Social Weather Stations for fielding my technical questions. He thanks, too, the Asian Barometer Project for providing access to its 2002 Philippine survey data files. The views expressed here are solely his own.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schaffer, F. Thin Descriptions: The Limits of Survey Research on the Meaning of Democracy. Polity 46, 303–330 (2014). https://doi.org/10.1057/pol.2014.14

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/pol.2014.14

Keywords

Search

Navigation