The “Dependent Variable Problem”: How Do We Know What Caused Desired Change?

Abstract

Governments typically introduce new programs because they want to achieve some sort of positive change. While policy makers may have a clear idea of what they want to achieve, there is often less certainty about the most effective way of achieving desired policy goals or even whether achieving a policy goal is possible given current financial constraints and/or institutional arrangements. Even when desired policy goals are achieved, it may be difficult to isolate which factor (or combination of factors) had the most significant impact. This difficulty is known as the dependent variable problem.

After introducing and defining the dependent variable problem, the chapter identifies the sort of research and policy questions that are explicitly concerned with causality and the different methods or techniques that policy makers can use to increase their understanding of what factor (or combination of factors) is facilitating, or impeding, desired change. The chapter discusses the strengths and limitations of randomized controlled trials where outcomes for two groups, one of whom underwent intervention, are compared, quasi-experimental approaches, and cross-case comparisons of sequential events. The chapter makes the point that, in the real world, causality is often complex, nonlinear, and/or influenced by macro-level factors beyond the control of policy makers responsible for a particular policy or program. For this reason, policy makers need to think carefully about the particular combination of techniques capable of generating the sort of information that will help them understand causal linkages.