Abstract
Social scientists often face a fundamental problem: Did I leave something causally important out of my explanation? How do I diagnose this? Where do I look for solutions to this problem? We build bridges between regression models and qualitative comparative analysis by comparing diagnostics and solutions to the problem of omitted variables and conditions. We then discuss various approaches and tackle the theoretical issues around causality which must be addressed before attending to technical fixes. In the conclusions, we reflect on the bridges built between the two traditions and draw more general lessons about the logic of social science research.
Similar content being viewed by others
Notes
In QCA, the term of ‘omitted variables’ is not used, also because it is inappropriate to speak about ‘variables’ (see fn. 2). However, for reasons of readability, we use this expression not only for our discussion on regression, but also when we deal with QCA.
This is more than just a linguistic matter of taste. Set-theoretic methods, such as QCA, do not look at variables but model various degrees of case membership in sets. The difference between the two understandings does not so much result in different values of the variable or the set membership, respectively, but lies in the perspective: variables look at a characteristics, while sets group cases according to a given case property. Sets, therefore, are very closely linked to concepts.
An instrumental variable has first and foremost a technical definition, contrary to the conceptual approach we followed when dealing with the proxy.
The tilde indicates the negation of a condition.
The superficiality consists in the fact that our description simplifies actual XY plots a bit. There can also be inconsistent cases in the upper right and lower left corner, namely those below the diagonal. However, for the argument made here, this is not important.
Since the extreme situation is already that every case has its ‘own’ explanation, there cannot be more explanatory paths than cases, unless we allow for values of ‘unique coverage’ being zero.
Another source for this phenomenon might be the generally low number of cases.
One may ask at this point whether it is then justified to lower expectations for large N QCAs and whether a low-expectation large N QCA is really desirable.
References
Angrist, J.D., and A.B. Krueger. 2001. Instrumental variables and the search for identification: From supply and demand to natural experiments. Journal of Economic Perspectives 15(4): 69–85.
Bennett, A., and J.T. Checkel. 2015. Process tracing: From philosophical roots to best practices. In Process tracing: From metaphor to analytical tool, ed. A. Bennett and J.T. Checkel, 3–38. Cambridge: Cambridge University Press.
Blanchard, O., and F. Giavazzi. 2003. Macroeconomic effects of regulation and deregulation in goods and labor markets. Quarterly Journal of Economics 118(3): 879–907.
Calabresi, G., and D. Melamed. 1972. Property rules, liability rules and inalienability: One view of the cathedral. Harvard Law Review 85(6): 1089–1128.
Clarke, K.A. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 22(4): 341–352.
Clarke, K.A. 2009. Return of the phantom menace: Omitted variable bias in political research. Conflict Management and Peace Science 26(1): 46–66.
Collier, D. 2014. Comment: QCA should set aside the algorithms. Sociological Methodology 44(1): 122–126.
Djankov, S., C. McLiesh, and R.M. Ramalho. 2006. Regulation and growth. Economic Letters 92: 395–401.
Dougherty, C. 2011. Introduction to econometrics, 4th ed. Oxford: Oxford University Press.
Fiss, P., D. Sharapov, and L. Cronqvist. 2013. Opposites attract? Opportunities and challenges for integrating large-N QCA and econometric analysis. Political Research Quarterly 66(1): 191–197.
Gerring, J. 2014. Causal mechanisms: Yes, but…. Comparative Political Studies 43(11): 1499–1526.
Goertz, G. 2006. Assessing the trivialness, relevance, and relative importance of necessary and sufficient conditions in social science. Studies in Comparative International Development 41(2): 88–109.
Greene, W.H. 2003. Econometric analysis, 5th ed. Upper Saddle River, NJ: Pearson Education.
Grofman, B., and C.Q. Schneider. 2009. An introduction to crisp set QCA, with a comparison to binary logistic regression. Political Research Quarterly 62(4): 662–672.
Hug, S. 2013. Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2): 252–265.
Jalilian, H., C. Kirkpatrick, and D. Parker. 2007. The impact of regulation on economic growth in developing countries: A cross-country analysis. World Development 35: 87–103.
King, G., R.O. Keohane, and S. Verba. 1994. Designing social inquiry: Scientific inference in qualitative research. Princeton: Princeton University Press.
Maggetti, M., F. Gilardi, and C.M. Radaelli. 2013. Designing research in the social sciences. Los Angeles: Sage.
Merton, R.K. 1957. On sociological theories of the middle range. In On theoretical sociology. Five essays, old and new, ed. R.K. Merton, 39–72. New York: The Free Press.
Moses, J., and T. Knutsen. 2012. Ways of knowing: Competing methodologies in social and political research. Basingstoke: Palgrave Macmillan.
Most, B.A., and H. Starr. 1989. Inquiry logic and international politics. Columbia: University of South Carolina Press.
Munck, G.L. 2016. Assessing set-theoretic comparative methods: A tool for qualitative comparativists? Comparative Political Studies 49(6): 775–780.
Paine, J. 2016a. Set-theoretic comparative methods: Less distinctive than claimed. Comparative Political Studies 49(6): 703–741.
Paine, J. 2016b. Still searching for the value added: Persistent concerns about set-theoretic comparative methods. Comparative Political Studies 49(6): 793–800.
Radaelli, C.M., and F. De Francesco. 2007. Regulatory quality in Europe: Concepts, measures and policy processes. Manchester: MUP.
Ragin, C.C. 1987. The comparative method. Berkeley: The University of California Press.
Ragin, C.C. 1994. Constructing social research: The unity and diversity of method. Thousand Oaks: Pine Forge Press.
Ragin, C.C. 2000. Fuzzy-set social science. Chicago: Chicago University Press.
Ragin, C.C. 2008. Redesigning social inquiry: Fuzzy sets and beyond. Chicago: University of Chicago Press.
Rihoux, B., and C.C. Ragin (eds.). 2009. Configurational comparative methods: Qualitative comparative analysis and related techniques. Thousand Oaks: SAGE.
Rihoux, B., P. Álamos-Concha, D. Bol, A. Marx, and I. Rezsöhazy. 2013. From niche to mainstream? A comprehensive mapping of QCA applications in journal articles from 1984 to 2011. Political Research Quarterly 66(1): 175–184.
Schneider, C.Q. 2016. Real differences and overlooked similarities. Set-methods in comparative perspective. Comparative Political Studies 49(6): 781–792.
Schneider, C.Q., and I. Rohlfing. 2013. Combining QCA and process tracing in set-theoretic multi-method research. Sociological Methods and Research 42(4): 559–597.
Schneider, C.Q., and C. Wagemann. 2012. Set-theoretic methods for the social sciences. Cambridge: Cambridge University Press.
Schneider, C.Q., and C. Wagemann. 2016. Assessing ESA on what it is designed for: A reply to Cooper and Glaesser. Field Methods 28(3): 316–321.
Radaelli, C.M., and Schrefler, L. 2011. Deregulation. In International encyclopedia of political science, ed. B. Badie, D. Berg-Schlosser, and L. Morlino. London: Sage.
Seawright, J. 2005. Qualitative comparative analysis vis-à-vis regression. Studies in Comparative and International Development 40(1): 3–26.
Skaaning, S.E. 2011. Assessing the robustness of crisp-set and fuzzy-set QCA results. Sociological Methods and Research 40(2): 391–408.
Starr, H. 2005. Cumulation from proper specification: Theory, logic, research design, and ‘nice’ laws. Conflict Management and Peace Science 22(4): 353–363.
Thiem, A., M. Baumgartner, and D. Bol. 2016. Still lost in translation! A correction of three misunderstandings between configurational comparativists and regressional analysis. Comparative Political Studies 49(6): 742–774.
Vis, B. 2012. The comparative advantages of fsQCA and regression analysis for moderately large-N analyses. Sociological Methods and Research 41(1): 168–198.
Wagemann, C., J. Buche, and M.B. Siewert. 2016. QCA and business research: Work in progress or a consolidated agenda. Journal of Business Research 69(7): 2531–2540.
Woolridge, J.M. 2008. Introductory econometrics. Andover: Cengage Learning.
Acknowledgements
We would like to thank the reviewers and the EPS co-editor Daniel Stockemer for their comments and suggestions. We also thank Fabrizio Gilardi, Jonathan C. Kamkhaji, Gabriel Katz and the participants to the ‘cake for comments’ seminar series 2016–2017 at the University of Exeter. We are grateful to Francesca Farmer for checking style and grammar. Research for this project was funded by the ERC project 694632 Protego, Procedural Tools for Effective Governance, and by the Goethe University of Frankfurt.
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Radaelli, C.M., Wagemann, C. What did I leave out? Omitted variables in regression and qualitative comparative analysis. Eur Polit Sci 18, 275–290 (2019). https://doi.org/10.1057/s41304-017-0142-7
Published:
Issue Date:
DOI: https://doi.org/10.1057/s41304-017-0142-7