Skip to main content
SpringerLink
Log in

Modified Distribution-Free Goodness-of-Fit Test Statistic

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Covariance structure analysis and its structural equation modeling extensions have become one of the most widely used methodologies in social sciences such as psychology, education, and economics. An important issue in such analysis is to assess the goodness of fit of a model under analysis. One of the most popular test statistics used in covariance structure analysis is the asymptotically distribution-free (ADF) test statistic introduced by Browne (Br J Math Stat Psychol 37:62–83, 1984). The ADF statistic can be used to test models without any specific distribution assumption (e.g., multivariate normal distribution) of the observed data. Despite its advantage, it has been shown in various empirical studies that unless sample sizes are extremely large, this ADF statistic could perform very poorly in practice. In this paper, we provide a theoretical explanation for this phenomenon and further propose a modified test statistic that improves the performance in samples of realistic size. The proposed statistic deals with the possible ill-conditioning of the involved large-scale covariance matrices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Amemiya, Y., & Anderson, T. W. (1990). Asymptotic chi-square tests for a large class of factor analysis models. Annals of Statistics, 18, 1453–1463.

    Article  Google Scholar 

  • Bellman, R. E. (1960). Introduction to matrix analysis. New York: McGraw-Hill Book Company.

    Google Scholar 

  • Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Book  Google Scholar 

  • Boomsma, A., & Hoogland, J. J. (2001). The robustness of lisrel modeling revisited. In Structural equation modeling: Present and future: A Festschrift in Honor of Karl Jöreskog (pp. 139–168). Chicago: Scientific Software International.

  • Browne, M. W. (1982). Covariance structures. In D. M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    Article  PubMed  Google Scholar 

  • Browne, M. W., & Shapiro, A. (1987). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematical and Statistical Psychology, 41, 193–208.

    Article  Google Scholar 

  • Browne, M. W., & Shapiro, A. (2015). Comments on the asymptotics of a distribution-free goodness of fit test statistic. Psychometrika, 80, 196–199.

    Article  PubMed  Google Scholar 

  • Byrne, B. M. (2012). Choosing structural equation modeling computer software: Snapshots of lisrel, eqs, amos, and mplus. In R. H. Hoyle (Ed.), Handbook of structural equation modeling, chap. 19 (pp. 307–324). New York: Guilford Press.

    Google Scholar 

  • Chun, S. Y., & Shapiro, A. (2009). Normal versus noncentral chi-square asymptotics of misspecified models. Multivariate Behavioral Research, 44(6), 803–827.

    Article  PubMed  Google Scholar 

  • Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods and Research, 26(3), 329–367.

    Article  Google Scholar 

  • Hu, L., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362.

    Article  PubMed  Google Scholar 

  • Huang, Y., & Bentler, P. M. (2015). Behavior of asymptotically distribution free test statistics in covariance versus correlation structure analysis. Structural Equation Modeling, 22, 489–503.

    Article  Google Scholar 

  • Jennrich, R., & Satorra, A. (2013). The nonsingularity of \(\gamma \) in covariance structure analysis of nonnormal data. Psychometrika, 79, 51–59.

    Article  PubMed  Google Scholar 

  • Kano, Y. (2002). Variable selection for structural models. Journal of Statistical Planning and Inference, 108(1–2), 173–187.

    Article  Google Scholar 

  • Lee, S.-Y. (1990). Multilevel analysis of structural equation models. Biometrika, 77(4), 763–772.

    Article  Google Scholar 

  • McManus, D. A. (1991). Who invented local power analysis? Econometric Theory, 7, 265–268.

    Article  Google Scholar 

  • Shapiro, A. (1986). Asymptotic theory of overparametrized structural models. Journal of the American Statistical Association, 81, 142–149.

    Article  Google Scholar 

  • Tomarken, A., & Waller, N. G. (2005). Structural equation modeling as a data-analytic framework for clinical science: Strengths, limitations, and misconceptions. Annual Review of Clinical Psychology, 1, 31–65.

    Article  PubMed  Google Scholar 

  • Wu, Hao, & Browne, M. W. (2015). Quantifying adventitious error in a covariance structure as a random effect. Psychometrika, 80(3), 571–600.

    Article  PubMed  PubMed Central  Google Scholar 

  • Xu, J., & Mackenzie, G. (2012). Modelling covariance structure in bivariate marginal models for longitudinal data. Biometrika, 99(3), 649–662.

    Article  Google Scholar 

  • Yuan, K. H., & Bentler, P. M. (1998). Normal theory based test statistics in structural equation modelling. British Journal of Mathematical and Statistical Psychology, 51, 289–309.

    Article  PubMed  Google Scholar 

  • Yuan, K. H., & Bentler, P. M. (1999). F tests for mean and covariance structure analysis. Journal of Educational and Behavioral Statistics, 24(3), 225–243.

    Article  Google Scholar 

  • Yuan, K. H., Hayashi, K., & Bentler, P. M. (2007). Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses. Journal of Multivariate Analysis, 98(6), 1262–1282.

    Article  Google Scholar 

Download references

Acknowledgements

Funding was provided for the third author by National Science Foundation (Grant No. CMMI1232623).

Author information

Authors and Affiliations

  1. McDonough School of Business, Georgetown University, Washington, DC, 20057 , USA

    So Yeon Chun

  2. Department of Psychology, Ohio State University, Columbus, OH, 43210-1222, USA

    Michael W. Browne

  3. School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332-0205, USA

    Alexander Shapiro

Authors
  1. So Yeon Chun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael W. Browne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Shapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to So Yeon Chun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chun, S.Y., Browne, M.W. & Shapiro, A. Modified Distribution-Free Goodness-of-Fit Test Statistic. Psychometrika 83, 48–66 (2018). https://doi.org/10.1007/s11336-017-9574-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-017-9574-9

Keywords

Search

Navigation