Skip to main content
SpringerLink
Log in

A Primer for Interpreting and Designing Difference-in-Differences Studies in Higher Education Research

  • Living reference work entry
  • First Online:
Higher Education: Handbook of Theory and Research

Part of the book series: Higher Education: Handbook of Theory and Research ((HATR,volume 35))

Abstract

Though randomized control trials continue to serve as the “gold standard” of causal inference, they are neither feasible nor desirable in numerous instances. Even in the absence of randomized trials, higher education researchers have at their disposal several statistical tools for estimating causal relationships. One such method is difference-in-differences, a powerful and intuitive approach to causal evaluation that exploits variation in the timing and coverage of policies. The method lends itself well to studying higher education policies and initiatives, as these frequently diffuse over time and across space in ways that may permit for causal inference. Difference-in-differences has become one of the most widely used methods for causal inference in higher education research. We use this chapter to introduce new researchers to this method with an overview of difference-in-differences models, common threats to their validity, and robustness checks. We then present extensions of the method, including event study models and variation in treatment timing. We illustrate these methods throughout the chapter by analyzing the effect of hurricanes on enrollment at affected colleges using data from the Integrated Postsecondary Education Data System and provide Stata code for replication of the analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.

    Article  Google Scholar 

  • Andrews, R. J., DesJardins, S., & Ranchhod, V. (2010). The effects of the Kalamazoo promise on college choice. Economics of Education Review, 29(5), 722–737.

    Article  Google Scholar 

  • Angrist, J. (2004). American education research changes tack. Oxford Review of Economic Policy, 20(2), 198–212.

    Article  Google Scholar 

  • Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton: Princeton University Press.

    Book  Google Scholar 

  • Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.

    Article  Google Scholar 

  • Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2019). Synthetic difference in differences (No. w25532). National Bureau of Economic Research.

    Google Scholar 

  • Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. The Review of Economics and Statistics, 60(1), 47–57.

    Article  Google Scholar 

  • Athey, S., & Imbens, G. W. (2018). Design-based analysis in difference-in-differences settings with staggered adoption (No. w24963). National Bureau of Economic Research.

    Google Scholar 

  • Baker, D. J. (2019). Pathways to racial equity in higher education: Modeling the antecedents of state affirmative action bans. American Educational Research Journal. Advance online publication. https://doi.org/10.3102/0002831219833918.

    Article  Google Scholar 

  • Belasco, A. S., Rosinger, K. O., & Hearn, J. C. (2015). The test-optional movement at America’s selective liberal arts colleges: A boon for equity or something else? Educational Evaluation and Policy Analysis, 37(2), 206–223.

    Article  Google Scholar 

  • Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in- differences estimates? The Quarterly Journal of Economics, 119(1), 249–275.

    Article  Google Scholar 

  • Betts, J. R., & McFarland, L. L. (1995). Safe port in a storm: The impact of labor market conditions on community college enrollments. Journal of Human Resources, 30(4), 741–765.

    Article  Google Scholar 

  • Bielby, R. M., House, E., Flaster, A., & DesJardins, S. L. (2013). Instrumental variables: Conceptual issues and an application considering high school course taking. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. XXVIII, pp. 263–321). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Brewer, M., Crossley, T. F., & Joyce, R. (2018). Inference with difference-in-differences revisited. Journal of Econometric Methods, 7(1), 1–16.

    Article  Google Scholar 

  • Bumphus, W. G., & Royal, A. M. (2007). In our own words: Disaster recovery post-Katrina. In A. W. Villadsen & G. E. De los Santos (Eds.), Hoping for the best while preparing for the worst: Disasters, emergencies and the community college (pp. 24–30). Phoenix: League for Innovation in the Community College.

    Google Scholar 

  • Charles, K., Hurst, E., & Notowidigdo, M. (2018). Housing booms and busts, labor market opportunities, and college attendance. American Economic Review, 108(10), 2947–2994.

    Article  Google Scholar 

  • Conley, T. G., & Taber, C. R. (2011). Inference with “`difference in differences” with a small number of policy changes. The Review of Economics and Statistics, 93(1), 113–125.

    Article  Google Scholar 

  • Cortes, K. E. (2010). Do bans on affirmative action hurt minority students? Evidence from the Texas Top 10% Plan. Economics of Education Review, 29(6), 1110–1124.

    Article  Google Scholar 

  • Dache-Gerbino, A. (2018). College desert and oasis: A critical geographic analysis of local college access. Journal of Diversity in Higher Education, 11(2), 97–116.

    Article  Google Scholar 

  • Daw, J. R., & Hatfield, L. A. (2018). Matching and regression to the mean in difference-in-differences analysis. Health Services Research, 53(6), 4138–4156.

    Article  Google Scholar 

  • Deryugina, T., Kawano, L., & Levitt, S. (2018). The economic impact of Hurricane Katrina on its victims: Evidence from individual tax returns. American Economic Journal: Applied Economics, 10(2), 202–233.

    Google Scholar 

  • DeWaard, J., Curtis, K., & Fussell, E. (2016). Population recovery in New Orleans after Hurricane Katrina: Exploring the potential role of stage migration in migration systems. Population and Environment, 37(4), 449–463.

    Article  Google Scholar 

  • Dynarski, S. M. (2003). Does aid matter? Measuring the effect of student aid on college attendance and completion. American Economic Review, 93(1), 279–288.

    Article  Google Scholar 

  • Foote, A., & Grosz, M. (2019). The effect of local labor market downturns on postsecondary enrollment and program choice. Education Finance and Policy. Advance online publication.

    Google Scholar 

  • Freyaldenhoven, S., Hansen, C., & Shapiro, J. M. (2018). Pre-event trends in the panel event-study design (No. w24565). National Bureau of Economic Research.

    Google Scholar 

  • Furquim, F., & Glasener, K. M. (2017). A quest for equity? Measuring the effect of QuestBridge on economic diversity at selective institutions. Research in Higher Education, 58(6), 646–671.

    Article  Google Scholar 

  • Furuta, J. (2017). Rationalization and student/school personhood in US college admissions: The rise of test-optional policies, 1987 to 2015. Sociology of Education, 90(3), 236–254.

    Article  Google Scholar 

  • Fussell, E. (2015). The long-term recovery of New Orleans’ population after Hurricane Katrina. American Behavioral Scientist, 59(10), 1231–1245.

    Article  Google Scholar 

  • Galea, S., Brewin, C., Gruber, M., Jones, R., King, D., King, L., McNally, R., Ursano, R., Petukhova, M., & Kessler, R. (2007). Exposure to hurricane-related stressors and mental illness after Hurricane Katrina. Archives of General Psychiatry, 64(12), 1427–1434.

    Article  Google Scholar 

  • Garces, L. M. (2013). Understanding the impact of affirmative action bans in different graduate fields of study. American Educational Research Journal, 50(2), 251–284.

    Article  Google Scholar 

  • Garces, L. M., & Mickey-Pabello, D. (2015). Racial diversity in the medical profession: The impact of affirmative action bans on underrepresented student of color matriculation in medical schools. The Journal of Higher Education, 86(2), 264–294.

    Article  Google Scholar 

  • Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. (2016). Impact evaluation in practice. Washington, DC: The World Bank.

    Book  Google Scholar 

  • Goodman-Bacon, A. (2018). Difference-in-differences with variation in treatment timing (No. w25018). National Bureau of Economic Research.

    Google Scholar 

  • Groen, J., & Polivka, A. (2008). The effect of Hurricane Katrina on the labor market outcomes of evacuees. American Economic Review, 98(2), 43–48.

    Article  Google Scholar 

  • Heckman, J. J., & Smith, J. A. (1999). The pre-programme earnings dip and the determinants of participation in a social programme: Implications for simple programme evaluation strategies. The Economic Journal, 109(457), 313–348.

    Article  Google Scholar 

  • Heller, D. E. (1999). The effects of tuition and state financial aid on public college enrollment. The Review of Higher Education, 23(1), 65–89.

    Article  Google Scholar 

  • Hemelt, S., & Marcotte, D. (2011). The impact of tuition increases on enrollment at public colleges and universities. Educational Evaluation and Policy Analysis, 33(4), 435–457.

    Article  Google Scholar 

  • Hillman, N. W. (2017). Geospatial analysis in higher education research. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. XXXII, pp. 529–575).

    Chapter  Google Scholar 

  • Hillman, N. W., & Orians, E. L. (2013). Community colleges and labor market conditions: How does enrollment demand change relative to local unemployment rates? Research in Higher Education, 54(7), 765–780.

    Article  Google Scholar 

  • Hillman, N. W., Tandberg, D. A., & Gross, J. P. K. (2014). Performance funding in higher education: Do financial incentives impact college completions? The Journal of Higher Education, 85(6), 826–857.

    Article  Google Scholar 

  • Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of “new” performance funding in higher education. Educational Evaluation and Policy Analysis, 37(4), 501–519.

    Article  Google Scholar 

  • Hillman, N. W., Fryar, A. H., & Crespín-Trujillo, V. (2018). Evaluating the impact of performance funding in Ohio and Tennessee. American Educational Research Journal, 55(1), 144–170.

    Article  Google Scholar 

  • Holland, P. W. (1986). Statistics and causal inference. Journal of the American statistical Association, 81(396), 945–960.

    Article  Google Scholar 

  • Hubbard, D. (2018). The impact of local labor market shocks on college choice: Evidence from plant closings in Michigan. Unpublished manuscript. Retrieved from https://www.aeaweb.org/conference/2019/preliminary/paper/Q67dtN77

  • Iacus, S. M., King, G., & Porro, G. (2012). Causal inference without balance checking: Coarsened exact matching. Political Analysis, 20(1), 1–24.

    Article  Google Scholar 

  • Jaquette, O., Kramer, D. A., & Curs, B. R. (2018). Growing the pie? The effect of responsibility center management on tuition revenue. The Journal of Higher Education, 89(5), 637–676.

    Article  Google Scholar 

  • Jarvie, J., & Landsberg, M. (2018, September 17). When a Category None hurricane is worse than a Category 5. Los Angeles Times. Retrieved from https://www.latimes.com

  • Johnson, G., & Rainey, S. (2007). Hurricane Katrina’s impact on three historically black colleges and universities (HBCUs): Voices from displaced students. Race, Gender & Class, 14(1), 100–119.

    Google Scholar 

  • Kahn-Lang, A., & Lang, K. (2019). The promise and pitfalls of differences-in-differences: Reflections on 16 and pregnant and other applications. Journal of Business & Economic Statistics, 1–14.

    Google Scholar 

  • Kates, R. W., Colten, C. E., Laska, S., & Leatherman, S. P. (2006). Reconstruction of New Orleans after Hurricane Katrina: a research perspective. Proceedings of the National Academy of Sciences, 103(40), 14653–14660.

    Article  Google Scholar 

  • Kothari, S. P., & Warner, J. B. (2007). Econometrics of event studies. In B. E. Eckbo (Ed.), Handbook of empirical corporate finance (pp. 3–36). North Holland: Elsevier.

    Chapter  Google Scholar 

  • Ladd, A., Gill, D., & Marszalek, J. (2007). Riders from the storm: Narratives of relocated New Orleans college students in the aftermath of Hurricane Katrina. Journal of Public Management & Social Policy, 13(2), 51–80.

    Google Scholar 

  • Lafortune, J., Rothstein, J., & Schanzenbach, D. W. (2018). School finance reform and the distribution of student achievement. American Economic Journal: Applied Economics, 10(2), 1–26.

    Google Scholar 

  • Lall, R. (2016). How multiple imputation makes a difference. Political Analysis, 24(4), 414–433.

    Article  Google Scholar 

  • Lapid, P. (2018). Expanding college access: The impact of new universities on local enrollment. Unpublished manuscript. Retrieved from https://www.ocf.berkeley.edu/~palapid/wp-content/uploads/2019/03/newU-SEA-18_11_05.pdf

  • Lechner, M. (2010). The estimation of causal effects by difference-in-difference methods. Foundations and Trends in Econometrics, 4(3), 165–224.

    Article  Google Scholar 

  • Li, A. Y., & Kennedy, A. I. (2018). Performance funding policy effects on community college outcomes: Are short-term certificates on the rise? Community College Review, 46(1), 3–39.

    Article  Google Scholar 

  • Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13–22.

    Article  Google Scholar 

  • Long, B. T. (2011). The new financial aid policies: Their impact on access and equity for low-income students. In L. M. Stulberg & S. L. Weinberg (Eds.), Diversity in American higher education: Toward a more comprehensive approach. New York: Routledge Books.

    Google Scholar 

  • Lowe, S., & Rhodes, J. (2012). Community college re-enrollment after Hurricane Katrina. Journal of College Student Retention, 14(2), 227–249.

    Article  Google Scholar 

  • Mangan, K., & O’Leary, B. (2010). New Orleans colleges 5 years after Katrina. The Chronicle of Higher Education. https://www.chronicle.com/article/New-Orleans-Colleges-5-Years/124107. Accessed 29 May 2019.

  • McCall, B. P., & Bielby, R. M. (2012). Regression discontinuity design: Recent developments and a guide to practice for researchers in higher education. In J. C. Smart & M. B. Paulsen (Eds.), Higher education: Handbook of theory and research (Vol. XXVII, pp. 249–290). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Murnane, R., & Willett, J. (2011). Methods matter: Improving causal inference in educational and social science research. Oxford, UK: Oxford University Press.

    Google Scholar 

  • Ngo, F., & Astudillo, S. (2019). California DREAM: The impact of financial aid for undocumented community college students. Educational Researcher, 48(1), 5–18.

    Article  Google Scholar 

  • Reynolds, C. L., & DesJardins, S. L. (2009). The use of matching methods in higher education research: Answering whether attendance at a 2-year institution results in differences in educational attainment. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXIV, pp. 47–97). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Rios-Aguilar, C., & Titus, M. (2018). Spatial research in higher education: Expanding understanding and identifying inequities. New Directions in Institutional Research, 2018(180), 5–10.

    Article  Google Scholar 

  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688.

    Article  Google Scholar 

  • Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469), 322–331.

    Article  Google Scholar 

  • Sacerdote, B. (2012). When the saints go marching out: Long-term outcomes for student evacuees from Hurricanes Katrina and Rita. American Economic Journal: Applied Economics, 4(1), 109–135.

    Google Scholar 

  • Savoie, E. J., O’Neil, R. M., & Rabban, D. M. (2007). Report of an AAUP special committee: Hurricane Katrina and New Orleans universities. Academe, 93(3), 59–126.

    Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  • Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103(484), 1334–1344.

    Article  Google Scholar 

  • St. Clair, T., & Cook, T. D. (2015). Difference-in-differences methods in public finance. National Tax Journal, 68(2), 319–338.

    Article  Google Scholar 

  • Stange, K. (2015). Differential pricing in undergraduate education: Effects on degree production by field. Journal of Policy Analysis and Management, 34(1), 107–135.

    Article  Google Scholar 

  • Strauss, V. (2018, September 4). The real story of New Orleans and its charter schools. The Washington Post. Retrieved from https://www.washingtonpost.com

  • Strezhnev, A. (2018). Semiparametric weighting estimators for multi-period difference-in-differences designs. Working paper. Retried from https://www.antonstrezhnev.com/research

  • Tandberg, D. A., & Hillman, N. W. (2014). State higher education performance funding: Data, outcomes, and policy implications. Journal of Education Finance, 29(3), 222–243.

    Google Scholar 

  • Turley, R. (2009). College proximity: Mapping access to opportunity. Sociology of Education, 82(2), 126–146.

    Article  Google Scholar 

  • U.S. Census Bureau. (2018a). Metropolitan and micropolitan: About. Retrieved from https://www.census.gov/programs-surveys/metro-micro/about.html

  • U.S. Census Bureau. (2018b). Metropolitan and micropolitan: Guidance for data users. Retrieved from https://www.census.gov/programs-surveys/metro-micro/guidance.html

  • U.S. Census Bureau. (2019). Resident population in Orleans Parish, LA [LAORLE0POP]. Retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/LAORLE0POP. 13 Apr 2019.

  • U.S. Department of Education. (2019). Integrated postsecondary education data system. https://nces.ed.gov/ipeds/datacenter/Data.aspx

  • U.S. Department of Homeland Security. (2019). OpenFEMA dataset: Disaster declarations summaries – V1. Retrieved from https://www.fema.gov/openfema-dataset-disaster-declarations-summaries-v1

  • Vigdor, J. (2008). The economic aftermath of Hurricane Katrina. Journal of Economic Perspectives, 22(4), 135–154.

    Article  Google Scholar 

  • Wolfers, J. (2006). Did unilateral divorce laws raise divorce rates? A reconciliation and new results. American Economic Review, 96(5), 1802–1820.

    Article  Google Scholar 

  • Zhang, L. (2010). The use of panel data models in higher education policy studies. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXV, pp. 307–349). Dordrecht: Springer.

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for the Study of Higher and Postsecondary Education, School of Education, University of Michigan, Ann Arbor, MI, USA

    Fernando Furquim

  2. University of Wisconsin-Madison, Madison, MI, USA

    Daniel Corral & Nicholas Hillman

Authors
  1. Fernando Furquim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Corral
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicholas Hillman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Furquim .

Editor information

Editors and Affiliations

  1. Graduate School of Education, University of Pennsylvania, Philadelphia, PA, USA

    Laura W. Perna

Section Editor information

  1. No affiliation provided

    Laura W. Perna

  2. No affiliation provided

    Ann Austin

  3. No affiliation provided

    Linda Eisenmann

  4. No affiliation provided

    Pamela Eddy

  5. No affiliation provided

    Adrianna Kezar

  6. No affiliation provided

    Anne-Marie Nunez

  7. No affiliation provided

    Shouping Hu

  8. No affiliation provided

    Anna Neumann

  9. No affiliation provided

    Nicholas A Bowman

  10. No affiliation provided

    Marvin Titus

  11. No affiliation provided

    Nicholas Hillman

Appendix Code

Appendix Code

The following Stata code produced many of the tables and charts included in this chapter. Because of space constraints, we have omitted various parts of the code such as data management steps and much of the analysis beyond Hurricane Katrina; we are happy to share complete .do files by request.

// open data for analysis use C:\Desktop\did_handbook_data.dta// install the following user-written programs: ssc install lgraph, replace ssc install blindschemes, replace ssc install estout, replace ssc install coefplot, replace// figure 1: fall enrollment for colleges in new orleans lgraph total_enroll year if treat==1, scheme(plottig) ylabel(0(500)3500) /// legend(off) loptions(1 lcolor(black) lpat(line) mcolor(black)) ytitle("Mean fall enrollment") xtitle("")// figure 2: fall enrollment for colleges in new orleans and other southern states lgraph total_enroll year treat, scheme(plottig) ylabel(0(500)3500) /// legend(on order(1 "Southern cities" 2 "New Orleans") position(6)) /// loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) ///lpat(line) mcolor(black)) ytitle("Mean fall enrollment") xtitle("")// table 3: did means table table treat post if (comparison==1 | treat==1), c(mean total_enroll) f(%10.0fc)// table 4: canonical did regression with different standard errors and covariates ∗ top panel: canonical did regression with different standard errors reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1) estimates store table4_a // no se adjustment reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), robust estimates store table4_b // robust s.e. reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), /// cluster(unitid) estimates store table4_c // cluster s.e. ∗ bottom panel: canonical did regression (cluster s.e.) with covariates and fe global controls "tuition1 metro_ue_rate" reg total_enroll i.treat i.post i.treat#i.post $controls if (comparison==1|treat==1), /// cluster(unitid) estimates store table4_d // controls only areg total_enroll i.treat i.post i.treat#i.post i.year if _est_table4_d==1 & /// (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table4_e // fixed effects only areg total_enroll i.treat i.post i.treat#i.post $controls i.year if _est_table4_d==1 &/// (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table4_f // controls and fe estout table4_a table4_b table4_c table4_d table4_e table4_f, cells(b(star fmt(2) /// label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) /// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// table 5: canonical did with multiple comparison groups, without and with controls ∗ create regional neighbor gen neighbor = 1 if inlist(stabbr,"AR","MS","TX") & sreb==1 ∗ create ps matched logit treat total_enroll if sreb==1 & year<2005 predict double ps ssc install psmatch2, replace psmatch2 treat, outcome(total_enroll) pscore(ps) egen ps_match = min(_weight), by(unitid) ∗ top panel (without controls) reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), /// cluster(unitid) estimates store table5_a // new orleans vs sreb reg total_enroll i.treat i.post i.treat#i.post, cluster(unitid) estimates store table5_b // new orleans vs nationwide reg total_enroll i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), /// cluster(unitid) estimates store table5_c // new orleans vs neighbor reg total_enroll i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), /// cluster(unitid) estimates store table5_d // new orleans vs matched ∗ bottom panel (with controls) reg total_enroll i.treat i.post i.treat#i.post $controls if (comparison==1|treat==1),/// cluster(unitid) estimates store table5_e // new orleans vs sreb reg total_enroll i.treat i.post i.treat#i.post $controls, cluster(unitid) estimates store table5_f // new orleans vs nationwide reg total_enroll i.treat i.post i.treat#i.post $controls if (neighbor==1|treat==1),///cluster(unitid) estimates store table5_g // new orleans vs neighbor reg total_enroll i.treat i.post i.treat#i.post $controls if (ps_match==1|treat==1),/// cluster(unitid) estimates store table5_h // new orleans vs matched estout table5_a table5_b table5_c table5_d table5_e table5_f table5_g table5_h, ///cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels///( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// figure 6: enrollment trend for multiple comparison groups lgraph total_enroll year treat if (sreb==1 | treat==1), scheme(plottig) /// ylabel(0(500)3500) name(sreb, replace) legend(on order(1 "SREB" 2 "New Orleans") ///position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)///mcolor(black)) ytitle("Mean fall enrollment") xtitle("") lgraph total_enroll year treat, scheme(plottig) ylabel(0(500)3500) name(us, replace) ///legend(on order(1 "U.S." 2 "New Orleans") position(6)) loptions(0 lcolor(black) ///lpat(dash) m(square); 1 lcolor(black) lpat(line) mcolor(black)) ytitle("Mean fall ///enrollment") xtitle("") lgraph total_enroll year treat if sreb==1 & (neighbor==1 | treat==1), scheme(plottig) ///ylabel(0(500)3500) name(neigh, replace) legend(on order(1 "Neighbors" 2 "New Orleans")///position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)///mcolor(black)) ytitle("Mean fall enrollment") xtitle("") lgraph total_enroll year treat if sreb==1 & (ps_match==1 | treat==1), scheme(plottig)///ylabel(0(500)4500) name(psm, replace) legend(on order(1 "PS Matched" 2 "New Orleans")///position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)///mcolor(black)) ytitle("Mean fall enrollment") xtitle("") graph combine sreb us neigh psm, name(combined, replace)// table 6: did regression with state-specific trends encode stabbr, gen(stn) areg total_enroll i.treat i.post i.treat#i.post i.stn##c.year if (comparison==1 | treat==1), absorb(stn) cluster(unitid) estimates store table6_a // state x year trends areg total_enroll i.treat i.post i.treat#i.post i.year if (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table6_b // fe only areg total_enroll i.treat i.post i.treat#i.post i.stn##c.year i.year if (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table6_c //state x years trends and fe estout table6_a table6_b table6_c, cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))//table 7: placebo test for change to treatment timing ∗generate placebo years gen placebo_2003 = 1 if year>=2003 gen placebo_2004 = 1 if year>=2004 gen placebo_2005 = 1 if year>=2005 gen placebo_2006 = 1 if year>=2006 gen placebo_2007 = 1 if year>=2007 recode placebo_2003-placebo_2007 (.=0) ∗ top panel (without controls) reg total_enroll i.treat i.placebo_2003 i.treat#i.placebo_2003 if year<2005, /// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect estimates store p_2003_noco reg total_enroll i.treat i.placebo_2004 i.treat#i.placebo_2004 if year<2005, /// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2004_noco reg total_enroll i.treat i.placebo_2005 i.treat#i.placebo_2005, cluster(unitid) est sto p_2005_noco reg total_enroll i.treat i.placebo_2006 i.treat#i.placebo_2006 if year>2005, /// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2006_noco reg total_enroll i.treat i.placebo_2007 i.treat#i.placebo_2007 if year>2005, /// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2007_noco ∗ bottom panel (with control) reg total_enroll i.treat i.placebo_2003 i.treat#i.placebo_2003 $controls if year<2005,///cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2003_co reg total_enroll i.treat i.placebo_2004 i.treat#i.placebo_2004 $controls if year<2005,/// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2004_co reg total_enroll i.treat i.placebo_2005 i.treat#i.placebo_2005 $controls, cluster(unitid) est sto p_2005_co reg total_enroll i.treat i.placebo_2006 i.treat#i.placebo_2006 $controls if year>2005,/// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2006_co reg total_enroll i.treat i.placebo_2007 i.treat#i.placebo_2007 $controls if year>2005,/// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2007_co estout p_2003_noco p_2004_noco p_2005_noco p_2006_noco p_2007_noco,cells(b(star fmt(2)///label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010)/// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) estout p_2003_co p_2004_co p_2005_co p_2006_co p_2007_co, cells(b(star fmt(2) ///label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010)///stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// table 8: placebo test for non-equivalent outcome ∗ top panel (in-state tuition) reg tuition1 i.treat i.post i.treat#i.post if (comparison==1 | treat==1), cluster(unitid)estimates store table8_a // new orleans vs sreb reg tuition1 i.treat i.post i.treat#i.post, cluster(unitid) estimates store table8_b // new orleans vs nationwide reg tuition1 i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), cluster(unitid) estimates store table8_c // new orleans vs neighbor reg tuition1 i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), cluster(unitid) estimates store table8_d // new orleans vs matched ∗ bottom panel (out-of-state tuition) reg tuition3 i.treat i.post i.treat#i.post if (comparison==1 | treat==1), cluster(unitid) estimates store table8_e // new orleans vs sreb reg tuition3 i.treat i.post i.treat#i.post, cluster(unitid) estimates store table8_f // new orleans vs nationwide reg tuition3 i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), cluster(unitid) estimates store table8_g // new orleans vs neighbor reg tuition3 i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), cluster(unitid) estimates store table8_h // new orleans vs matched estout table8_a table8_b table8_c table8_d table8_e table8_f table8_g table8_h,/// cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels///( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// table 9: did regression for multiple hurricanes areg total_enroll i.treat_mh i.year if inc==1, vce(cluster unitid) absorb(unitid)// table 10: event study results (hurricane katrina) ∗ create adoption year of treatment and limit to five-year pre/post period gen adopt_delta=2005-year if treat==1 gen within_5=(adopt_delta==. | inrange(adopt_delta, -5, 5)) ∗create event study lag (here’s how to do it in a loop) forvalues i=1(1)5 { gen predelta_`i'=(adopt_delta==`i') label var predelta_`i' "-`i'" } ∗ create event study lead (here’s how to do it one by one) gen postdelta_0=(adopt_delta==0) label var postdelta_0 "0" gen postdelta_1=(adopt_delta==-1) label var postdelta_1 "1" gen postdelta_2=(adopt_delta==-2) label var postdelta_2 "2" gen postdelta_3=(adopt_delta==-3) label var postdelta_3 "3" gen postdelta_4=(adopt_delta<=-4) label var postdelta_4 "4" gen postdelta_5=(adopt_delta<=-5) label var postdelta_5 "5" areg total_enroll predelta_5 predelta_4 predelta_3 predelta_2 postdelt∗ i.year if ///within_5==1 & year<=2010, vce(cluster unitid) absorb(unitid) estimates store fig7// figure 7: event study estimates (hurricane katrina) coefplot fig7, keep ( predelta_5 predelta_4 predelta_3 predelta_2 postdelta_0 ///postdelta_1 postdelta_2 postdelta_3 postdelta_4 postdelta_5) vertical xlabel ///(, angle(vertical)) xtitle("Years since hurricane (0)")ytitle("Estimated effect")///yline(0, lcolor(black)) scheme(plottig)// table 11: event study results (all hurricanes) ∗ create new within_5 & pre/post indicators because they should be relative to each hurricane cap drop adopt_delta within_5 predelta∗ postdelta∗ gen adopt_delta=year-first_hurr if hurr_ever==1 ∗create event study lag (here’s how to do it in a loop) forvalues i=1(1)5 { gen predelta_`i'=(adopt_delta==`i') label var predelta_`i' "-`i'" } ∗ create event study lead (here’s how to do it one by one) gen postdelta_0=(adopt_delta==0) label var postdelta_0 "0" gen postdelta_1=(adopt_delta==-1) label var postdelta_1 "1" gen postdelta_2=(adopt_delta==-2) label var postdelta_2 "2" gen postdelta_3=(adopt_delta==-3) label var postdelta_3 "3" gen postdelta_4=(adopt_delta<=-4) label var postdelta_4 "4" gen postdelta_5=(adopt_delta<=-5) label var postdelta_5 "5" ∗limit to obs within 5 yrs of a hurricane. gen within_5=(adopt_delta==. | inrange(adopt_delta, -5, 5)) ∗ coefficient only areg total_enroll predelta_2 predelta_3 predelta_4 predelta_5 postdelt∗ i.year if ///within_5==1 & inc==1, vce(cluster unitid) absorb(unitid) ∗ adding state-specific trends areg total_enroll predelta_2 predelta_3 predelta_4 predelta_5 postdelt∗ i.year ///i.year##c.stn if within_5==1 & inc==1, vce(cluster unitid) absorb(unitid)// figure 8: event study estimates (all hurricanes) coefplot fig8a, keep ( predelta_5 predelta_4 predelta_3 predelta_2 postdelta_0 /// postdelta_1 postdelta_2 postdelta_3 postdelta_4 postdelta_5) vertical xlabel ///(, angle(vertical)) xtitle("Years since hurricane (0)") ytitle("Estimated effect")///yline(0, lcolor(black)) scheme(plotplain)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Furquim, F., Corral, D., Hillman, N. (2020). A Primer for Interpreting and Designing Difference-in-Differences Studies in Higher Education Research. In: Perna, L. (eds) Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research, vol 35. Springer, Cham. https://doi.org/10.1007/978-3-030-11743-6_5-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-11743-6_5-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-11743-6

  • Online ISBN: 978-3-030-11743-6

  • eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education

Publish with us

Policies and ethics

Search

Navigation