Statistical Sophistry

Abstract

Since the late 1940s, statistical analysis has been increasingly applied to economics discipline. As a result, today’s macroeconometrics is essentially a synonym for macroeconomics. Is it sound to apply probability laws to macroeconomics? What is the consequence of econometricalization? What is the scientific way to conduct macroeconomic research? This chapter attempts to answer these questions.

Appendix (for Section 3.6.2): Why Econometric Models Fail: An Illustration

To demonstrate the performance of macroeconometric models, the author uses the US macroeconomic time series data 1969–2013 (see Table 3.2 at the end of appendix) to estimate the GDP identity, i.e. the expenditure and income sides of GDP. Since there are some statistical discrepancies in the two sides of the GDP, one must choose one side as the GDP value. The author chooses the income side of the GDP. Based on the 1-lagged GDP and the variables on both sides of the GDP, the author uses the OLS method to estimate 7 models. Here, the author has an upper hand over even the most experienced macroeconometricians and can dismiss any criticism on modelling techniques because we have a complete list of all factors and know the true mechanism (the correct function of the GDP equation). The estimated results for various models are shown in Table 3.1. For the benefit of non-econometricians, the author has not only listed the coefficient and standard error for each variable but also listed the p-value.

Table 3.1 Estimation of GDP determinants based on US time series 1969–2013

Table 3.2 The expenditure and income sides of US GDP 1989–2013

Model 1 use income-side data to estimate the GDP identity. The estimated results for Model 1 are perfect: the adjusted R-squared is 1, all variables on the income side of the GDP are extremely significant (p=0.000) and the coefficients are extremely close to 1. The coefficient for the constant is close to zero with a very high p-value (p=0.732), indicating there is no constant term in the model. These are exactly what is predicted by the GDP identity:

GDP=Wage +Tax + Profit + Capital Formation.

One may hail that the macroeconometric model works! However, this is not the usual case in macroeconometric modelling and the model has worked because the assumptions for estimating an econometric model held. With perfect theoretic knowledge we know that all variables are included in the model so the conditions for random experiments hold. We also know perfectly well that we have the right function for the model. More importantly, the data perfectly fit in with the GDP identity equation except for very tiny rounding errors (about US$0.1 billion for a magnitude of US$1018–16980 billion GDP) for some years, so the OLS method can find the best fit.

Model 2 uses the income-side GDP data to estimate the GDP identity on the expenditure side:

GDP= Consumption + Investment + Net Export + Government Spending.

With perfect knowledge, this model also includes all variables and uses the right function. However, the data do not fit the equation closely because of the statistic discrepancy (measurement error) on both sides of the GDP. The measurement error causes much damage to the estimation. Although R-squared is still very high (0.9999) and most explanatory variables are significant, the results are quite far from the truth: all coefficients are not close to 1. The marginal contribution of consumption is overestimated while the marginal contribution of investment and net export is underestimated; the marginal contribution of government spending to the GDP is only about 16%. Compared with the true marginal contribution of 100% based on our perfect knowledge, the estimated marginal contribution of government spending discounts the true value by more than 80%. Effectively, the marginal contribution of government spending is insignificant even if one uses the 10% p-value as a benchmark of rejection of significance of government spending. Moreover, the constant should be zero but modelling results show it is very significant.

Model 3 estimates the impact of a lagged GDP on current GDP to illustrate the common practice of using lagged variables in macroeconometric modelling. The estimation shows a very high R-squared (0.9984) and a very significant impact of past GDP. In fact, the coefficient of 1-lagged GDP is close to (or slightly greater than) 1. This confirms the view that most macroeconomic variables are non-stationary. However, the unit root tests on GDP and other variables are mixed, depending on what type of test is employed. If one believes these variables are non-stationary and thus he/she employs a first-differenced model or a cointegrated VAR model, the results may interest a macroeconometrician but this approach is definitely a step further on the wrong way to finding the truth because there are no dynamics in the GDP identity equation.

Model 4 includes all variables from both sides of the GDP. This exercise assumes that we have no knowledge of what variable is relevant or important so we have to include all possible variables. The estimated results show that the coefficients on the income-side variables are very close to 1 while those on the expenditure-side variables are very close to zero. Since the coefficients on the income-side variables are quite close to the results in Model 1, one may conclude that the irrelevant variables added to the model will not change the modelling results. However, here the expenditure-side variables are not irrelevant variables—they are components of the GDP! Their coefficients are zero simply because the model has already found the best fit, so they become redundant variables. This reasoning is confirmed by the fact that when the expenditure side of the GDP values are used as the values for dependent variables, the expenditure side of the GDP components become very significant (with coefficients close to 1), while the income side of the GDP components is insignificant. Hence, these results demonstrate that an econometric model cannot find which variables are relevant or important but can only suggest which variables can fit the data better.

Models 5–7 show different combinations of variable selections. Model 5 includes the 1-lagged GDP and expenditure-side variables. The results show that the coefficients for the expenditure-side variables are very similar to the results from Model 2, while the lagged GDP becomes insignificant. Again, this result does not indicate that the expenditure-side variables are more important than the lagged GDP, but only shows that the expenditure-side variable can fit the data better than the lagged GDP. Model 6 keeps the relatively more important variables on the expenditure side—wages and profits—but excludes the relatively less important variables—taxes and fixed capital formations. The results show the significant overstatement of the contribution of wages and profits. This is simply the consequence of omitting variables in macroeconometric models, but this model represents a likely case in macroeconometric modelling because in real econometric modelling practice no one has perfect knowledge to include all variables. Model 7 includes the most important variables from both sides of the GDP, namely wages, profits, consumption and investment. The estimation results do not make sense in economics: wages, profits and consumption make a discounted contribution to GDP (the coefficients for these variables are significantly less than 1) while investment contributes negatively (albeit insignificantly) to the GDP.

From this exercise of illustrative estimation, it is seen that, if the conditions for statistical theory hold, a statistical model works well (e.g. Model 1). However, this is an unlikely case in macroeconometrics because we have neither perfect knowledge about the factors involved nor the correct functions to be used, and also because the macroeconomic data are not accurate. From the performance of Models 5–7, we can see how misleading a macroeconometric model can be. Considering the possibility of misspecification of function forms in real econometric modelling practice, the estimation results can be even worse than the example displayed. In short, a macroeconometric model is most likely to be unable to find the truth due to measurement errors in data (e.g. Model 2), the inability to include all possible factors (e.g. Models 5, 6, 7), interference between explanatory variables (Models 5, 7) and misspecification of function form.