Abstract
Ordered rating scales are one of the most frequently used question formats in large-scale surveys. Analysts of the responses to such questions often find themselves in need of describing the degree of agreement (concentration, consensus) of the answers to such questions. For that purpose they commonly use standard deviations of the response distributions, or measures based on these (such as the coefficient of consensus defined by Granberg and Holmberg, 1988), or the coefficient of variability, etc. This paper demonstrates that such measures are inappropriate for this purpose because they misrepresent what they are supposed to measure: the `peakedness' of a distribution. As an alternative a measure of agreement A is proposed. This measure is a weighted average of the degree of agreement that exists in the simple component parts – layers – into which any frequency distribution can be disaggregated, and for which agreement can be expressed in a straightforward and unequivocal way.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brug, W. van der (1997). Where's the Party? Voters' Perceptions of Party Positions. Doctoral Dissertation: University of Amsterdam.
Brug, W. van der & van der Eijk, C. (1999). The cognitive basis of voting. In: H. Schmitt & J. J. A. Thomassen (eds), Political Representation and Legitimacy in the European Union. Oxford: Oxford University Press, pp. 129-160.
Converse, P. E. (1964). The nature of belief systems in mass publics. In: D. E. Apter (ed.), Ideology and Discontent. New York: Free Press, pp. 206-261.
Converse, P. E. (1975). Public opinion and voting behavior. In: F. I. Greenstein & N.W. Polsby (eds), Handbook of Political Science, Vol. 4, Reading, MA: Addison Wesley, pp. 75-169.
Converse, P. E. & Pierce, R. (1986). Political Representation in France. Cambridge, MA: Belknap Press.
Coombs, C. H. (1964). A Theory of Data. New York: Wiley.
Downs, A. (1957). An Economic Theory of Democracy. New York: Harper and Row.
Eijk, C. van der (1990). Ideology, party systems and voting in Europe. Paper presented at Annual Meeting of the American Political Science Association, San Francisco, August 1990.
Eijk, C. van der & Franklin, M. N. (1996). Choosing Europe? The European Electorate and National Politics in the Face of Union. Ann Arbor: University of Michigan Press.
Eijk, C. van der, Franklin, M. N. & van der Brug, W. (1999). Policy preferences and party choice. In: H. Schmitt & J. J. A. Thomassen (eds), Political Representation and Legitimacy in the European Union. Oxford: Oxford University Press, pp. 161-185.
Eijk, C. van der, Niemöller, B. & Eggen, A. Th. J. (1981). Dutch Parliamentary Election Study 1981. Amsterdam: University of Amsterdam.
Eijk, C. van der, Oppenhuis, E. V. & Schmitt, H. (1993). EES-89: European Election Study 1989. Data Description and Preliminary Documentation. Amsterdam: Steinmetz Archive/Swidoc.
Granberg, D. & Holmberg, S. (1988). The Political System Matters. Social Psychology and Voting Behavior in Sweden and the United States. Cambridge: Cambridge University Press.
Herrera, C., Herrera, R. & Smith, E. (1992). Public opinion and congressional representation. Public Opinion Quarterly 56: 185-205.
Huber, J. & Bingham Powell, G. (1994). Congruence between citizens and policymakers in two visions of liberal democracy. World Politics 46: 291-326.
McClosky, H. (1964). Consensus and ideology in American politics. American Political Science Review 58: 361-382.
McClosky, H., Hoffman, P. J. & O'Hara, R. (1960). Issue conflict and consensus among party leaders and followers. American Political Science Review 54: 406-427.
Schmitt, H., van der Eijk, C., Scholz, E. & Klein, M. (1996). European Election Study 1994. Data Description and Documentation. Mannheim/Cologne, Zentralarchiv für Empirische Sozialforschung/ European Election Study Research Group.
Schuur, W. H. van (1984). Structure in Political Beliefs. A New Model for Stochastic Unfolding with Applications to European Party Activists. Amsterdam: CT Press.
Schuur, W. H. van (1993). Nonparametric unfolding models for multicategory data. Political Analysis 4: 41-74.
Thomassen, J. J. A. & Jennings, M. K. (1989). Party Systems, Party Differences and Homogeneity. Paper presented at IPSA/ISAWorkgroup on Elections and Parties, Paris, April 1989.
Thomassen, J. J. A. & Schmitt, H. (1999). Partisan structures in the European parliament. In: R. S. Katz & B. Wessels (eds), The European Parliament, the National Parliaments and European Integration. Oxford: Oxford University Press, pp. 129-148.
Oppenhuis, E. V. (1995). Voting Behavior in Europe. A Comparative Analysis of Electoral Participation and Party Choice. Amsterdam: Het Spinhuis.
Pomper, G.M. (1972). From confusion to clarity: Issues and American voters, 1956–1968. American Political Science Review 66: 415-428.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Van Der Eijk, C. Measuring Agreement in Ordered Rating Scales. Quality & Quantity 35, 325–341 (2001). https://doi.org/10.1023/A:1010374114305
Issue Date:
DOI: https://doi.org/10.1023/A:1010374114305