Skip to main content

Advertisement

SpringerLink
Log in

Policy spillover effects on student achievement: evidence from PISA

  • Original Paper
  • Published:
Letters in Spatial and Resource Sciences Aims and scope Submit manuscript

Abstract

National education reforms do not occur in isolation. Countries look towards each other to identify ways that improve the quality of their education systems. When evaluating the effect of an education policy, it is worth considering both local effects of the policy and its spillover effects on other countries. Ignoring spillover effects between countries can lead to biased estimates of policy effects and suboptimal decision making. This paper examines spillover effects of one widespread education policy, school autonomy, on student achievement using three waves of data from the Programme for International Student Assessment (PISA). The spatial autoregressive model is applied to capture both spillover and local effects of school autonomy. Overall, school autonomy raises student achievement in Reading, Mathematics, and Science. We confirm the existence of positive and statistically significant average spillover effects; thus, estimates based on linear regression underestimate the impact of school autonomy. Our findings indicate that there is spatial dependence in student achievement across countries linked to the geographic proximity between countries. Possible extensions of this work are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Availability of data and material

Data used in this project are publicly available at https://www.oecd.org/pisa/data/

Code availability

All estimation was conducted using the open source software R. All codes are available from the authors.

Notes

  1. LeSage and Pace (2010) refers to local and spillover effects as direct and indirect effects—see also Fischer et al. (2009), LeSage and Pace (2010), and LeSage and Pace (2013).

  2. Teltemann and Windzio (2018) takes a first step in this direction by testing and controlling for spatial proximity between countries in an analysis of student performance. The authors do not, unlike this paper, quantify spillover effects.

  3. The process by which countries influence each others’ policy decisions is defined as policy borrowing, transfer, or diffusion—see, e.g., Meseguer and Gilardi (2009).

  4. Due to push-back from parents and teachers, the 2007 reform was revoked in 2013.

  5. In the online appendix, we explore other conditional factors such as GDP per capita, OECD membership, and income categories.

  6. Country-level summary statistics are provided in the online appendix.

  7. For variables included in our analyses, this constitutes less than 5% of student observations.

  8. Private school status values for Sweden and Israel were missing in 2015. These two missing values were imputed using the respective country averages of 2009 and 2012.

  9. For example, countries either have nationwide exit exams or they do not.

  10. To avoid perfect collinearity with the constant term, \(\eta _N\) and \(\delta _T\) are dropped.

  11. Correlated shocks arise when factors, e.g., a “PISA shock" or global economic shocks, cause countries to react similarly.

  12. \({\varvec{w}}_n\) is intransitive when a neighbour’s neighbour is not a neighbour.

  13. This is the type of interpretation found in Teltemann and Windzio (2018), and Gaku and Tsyawo (2021) as the authors consider spatially-lagged outcome as a control.

  14. A feedback effect occurs when an impact from a country passes through its neighbour back to the country of origin.

  15. The standard deviations of PISA test scores are given in Table 1.

  16. Partial effects are highly non-linear functions of the asymptotically normally distributed parameter estimates—see Sect. 3.2.

  17. E.g., for Reading, this value is computed as \(\frac{113.977/10}{44.004} \times 100\% \approx 25\%\) where the total effect is 113.977 and the standard deviation of reading scores is 44.004 (see Table 1).

  18. The results in Table 4 is based on Reading. See the online appendix for results by subdomain on Mathematics and Science.

  19. Robustness analyses for Mathematics and Science give similar results and are thus not provided in the main text—see the online appendix.

  20. This is likely attributable to the smaller sample size.

References

  • Ammermueller, A.: Institutional features of schooling systems and educational inequality: cross-country evidence from PIRLS and PISA. German Econ. Rev. 14(2), 190–213 (2013)

    Article  Google Scholar 

  • Anselin, L., Gallo, J.L., Jayet, H.: Spatial Panel Econometrics. The Econometrics of Panel Data. Springer, pp. 625–660 (2008)

  • Baird, J.-A.: et al. On the supranational spell of PISA in policy. Educ. Res. 58.2, 121–138 (2016)

  • Bramoullé, Y., Djebbari, H., Fortin, B.: Identification of peer effects through social networks. J. Econometr. 150(1), 41–55 (2009)

    Article  Google Scholar 

  • Bramoullé, Y., Djebbari, H., Fortin, B.: Peer Effects in Networks: A Survey” (2019)

  • Breakspear, S.: The Policy Impact of PISA: An Exploration of the Normative Effects of International Benchmarking in School System Performance. OECD (2012)

  • Cantley, I.: PISA and policy-borrowing: a philosophical perspective on their interplay in mathematics education. Educ. Philos. Theory 51(12), 1200–1215 (2019)

    Article  Google Scholar 

  • Coghlan, M., Desurmont, A.: School Autonomy in Europe Policies and Measures. Brussels (2007)

  • Fischer, M.M., Bartkowska, M., Riedl, A., Sardadvar, S., Kunnert, A.: The impact of human capital on regional labor productivity in Europe. Lett. Spatial Resour. Sci, 2.2-3, 97–108 (2009)

  • Forestier, K., Crossley, M.: International education policy transfer-borrowing both ways: the Hong Kong and England experience. Compare J. Comp. Int. Educ. 45.5, 664–685 (2015)

  • Fuchs, T., Wößmann, L.: What accounts for international differences in student performance? A re-examination using PISA data. The Economics of Education and Training. Springer, pp. 209–240 (2008)

  • Gaku, S., Tsyawo, Emmanuel, S.: Neighbourhood effects and the incidence of child labour. Lett. Spatial Resour. Sci. 14.3, pp. 247–259 (2021)

  • Gill, T., Benton, T.: Investigating the Relationship Between Aspects of Countries’ Assessment Systems and Achievement on the Programme for International Student As sessment (PISA) Tests. Cambridge Assessment, Cambridge (2013)

    Google Scholar 

  • Gray, J., Galton, M., Colleen, M.: Wellbeing and the Young Adolescent. Cambridge Scholars Publishing, The supportive school (2011)

  • Grek, S.: Governing by numbers: the PISA ‘effect’ in Europe. J. Educ. Policy 24(1), 23–37 (2009)

    Article  Google Scholar 

  • Gulson, K.N., Symes, C.: Spatial Theories of Education: Policy and Geography Matters, vol. 9. Routledge Research in Education. Routledge, New York (2007)

  • Hanushek, E.A., Wößmann, L.: Does educational tracking affect performance and inequality? Differences-in-differences evidence across countries. Econ. J. 116.510, C63–C76 (2006)

  • Hanushek, E.A., Link, S., Wößmann, L.: Does school autonomy make sense everywhere? Panel estimates from PISA. J. Dev. Econ. 104, 212–232 (2013)

    Article  Google Scholar 

  • Holzinger, K., Knill, C.: Causes and conditions of cross-national policy convergence. J. Eur. Publ. Policy 12(5), 775–796 (2005)

    Article  Google Scholar 

  • Kelejian, H.H., Prucha, I.R.: A generalized spatial two-stage least squares procedure for estimating a spatial autoregressive model with autoregressive disturbances. J. Real Estate Finance Econ. 17.1, 99–121 (1998)

  • König, M.D., Liu, X., Zenou, Y.: R &D networks: theory, empirics, and policy implications. Rev. Econ. Stat. 101.3, 476–491 (2019)

  • Larsen, M.A., Beech, J.: Spatial theorizing in comparative and international education research. Comp. Educ. Rev. 58(2), 191–214 (2014)

    Article  Google Scholar 

  • Lee, L.-F.: GMM and 2SLS estimation of mixed regressive, spatial autoregressive models. J. Econometr. 137(2), 489–514 (2007)

    Article  Google Scholar 

  • LeSage, J.P., Pace, R.K.: Spatial Econometric Models. Handbook of Applied Spatial Analysis. Springer, pp. 355–376 (2010)

  • LeSage, J.P., Pace, R.K.: Interpreting Spatial Econometric Models. Handbook of Regional Science. Springer, pp. 1535–1552 (2013)

  • LeSage, J.P., Pace, R.K.: Introduction to Spatial Econometrics. CRC Press (2009)

  • Lin, X.: Identifying peer effects in student academic achievement by spatial autoregressive models with group unobservables. J. Law Econ. 28(4), 825–860 (2010)

    Google Scholar 

  • Manski, C.F.: Identification of endogenous social effects: the reflection problem. Rev. Econ. Stud. 60(3), 531–542 (1993)

    Article  Google Scholar 

  • Maslowski, R., Scheerens, J., Luyten, H.: The effect of school autonomy and school internal decentralization on students’ reading literacy. Sch. Eff. Sch. Improv. 18(3), 303–334 (2007)

    Article  Google Scholar 

  • Mayer, T., Soledad, Z.: Notes on CEPII’s Distances Measures: The GeoDist Database. SSRN Electr. J. (2011)

  • Meseguer, C., Gilardi, F.: What is new in the study of policy diffusion? Rev. Int. Polit. Econ. 16(3), 527–543 (2009)

    Article  Google Scholar 

  • Obinger, H., Schmitt, C., Starke, P.: Policy diffusion and policy transfer in comparative welfare state research. Social Policy Admin. 47(1), 111–129 (2013)

    Article  Google Scholar 

  • OECD.: PISA 2015 Results (vol. I): Excellence and Equity in Education. OECD Publishing (2016)

  • Parcerisa, L., Clara, F., Antoni, V.: Understanding the PISA influence on national education policies: a focus on policy transfer mechanisms. International perspectives on school settings, education policy and digital strategies. A transatlantic discourse in education research, pp. 185–198 (2020)

  • Phillips, D., Ochs, K.: Processes of policy borrowing in education: some explanatory and analytical devices. Comp. Educ. 39(4), 451–461 (2003)

    Article  Google Scholar 

  • Plümper, T., Neumayer, E.: Model specification in the analysis of spatial dependence. Eur. J. Polit. Res. 49(3), 418–442 (2010)

    Article  Google Scholar 

  • Ringarp, J., Rothland, M.: Is the grass always greener? The effect of the PISA results on education debates in Sweden and Germany. Eur. Educ. Res. J. 9(3), 422–430 (2010)

    Article  Google Scholar 

  • Sacerdote, B.: Peer effects in education: How might they work, how big are they and how much do we know thus far? Handbook of the Economics of Education. Vol. 3. Elsevier, pp. 249–277 (2011)

  • Schneeweis, N., Winter-Ebmer, R.: Peer effects in Austrian schools. Emp. Econ. 32(2–3), 387–409 (2007)

    Article  Google Scholar 

  • Sellar, S., Thompson, G., David, R.: Taking the Measure of PISA and International Testing. Brush Education, The Global Education Race (2017)

  • Teltemann, J., Windzio, M.: The impact of marketisation and spatial proximity on reading performance: international results from PISA 2012. Compare J. Comp. Int. Educ. 49.5, 777–794 (2018)

  • Waldow, F.: Projecting images of the ‘good’and the ‘bad school’: Top scorers in educational large-scale assessments as reference societies. Compare J. Comp. Int. Educ. 47.5, pp. 647–664 (2017)

  • West, M.R., Woessmann, L.: ‘Every Catholic Child in a Catholic School’: historical resistance to state schooling, contemporary private competition and student achievement across countries. Econ. J. 120.546, F229–F255 (2010)

  • Wößmann, L.: International evidence on school competition, autonomy, and accountability: a review. Peabody J. Educ. 82(2–3), 473–497 (2007)

    Article  Google Scholar 

  • Wößmann, L.: The importance of school systems: evidence from international differences in student achievement. J. Econ. Perspect. 30(3), 3–32 (2016)

    Article  Google Scholar 

Download references

Funding

The authors did not receive any funding for the execution of this project.

Author information

Authors and Affiliations

  1. Graduate School of Education, University of Pennsylvania, 3700 Walnut St, Philadelphia, PA, 19104, USA

    Clara-Christina E. Gerstner

  2. AIRESS and FGSES, Université Mohammed VI Polytechnique, Rabat, Morocco

    Emmanuel S. Tsyawo

Authors
  1. Clara-Christina E. Gerstner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanuel S. Tsyawo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emmanuel S. Tsyawo.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest or competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

We thank Michael Rovine, Thanh Lu, Keisha Solomon, Eric Kadio, and Ashley Mcfarlane for valuable comments.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 988 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gerstner, CC.E., Tsyawo, E.S. Policy spillover effects on student achievement: evidence from PISA. Lett Spat Resour Sci 15, 523–541 (2022). https://doi.org/10.1007/s12076-022-00310-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12076-022-00310-y

Keywords

JEL Classification

Search

Navigation