Abstract
The method of maximum likelihood using the EM-algorithm for fitting finite mixtures of normal distributions is the accepted method of estimation ever since it has been shown to be superior to the method of moments. Recent books testify to this. There has however been criticism of the method of maximum likelihood for this problem, the main criticism being when the variances of component distributions are unequal the likelihood is in fact unbounded and there can be multiple local maxima. Another major criticism is that the maximum likelihood estimator is not robust. Several alternative minimum distance estimators have since been proposed as a way of dealing with the first problem. This paper deals with one of these estimators which is not only superior due to its robustness, but in fact can have an advantage in numerical studies even at the model distribution. Importantly, robust alternatives of the EM-algorithm, ostensibly fitting t distributions when in fact the data are mixtures of normals, are also not competitive at the normal mixture model when compared to the chosen minimum distance estimator. It is argued for instance that natural processes should lead to mixtures whose component distributions are normal as a result of the Central Limit Theorem. On the other hand data can be contaminated because of extraneous sources as are typically assumed in robustness studies. This calls for a robust estimator.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amemiya T (1985) Advanced econometrics. Harvard University Press, Cambridge
Benaglia T, Chauveau D, Hunter DR, Young D (2009) Mixtools: an R package for analysing finite mixture models. J Stat Soft 32(6):1–29
Basford KE, Greenway DR, McLachlan GJ, Peel D (1997) Standard errors of fitted means under normal mixture. Comput Stat 12:1–17
Biernacki C, Chretien S (2003) Degeneracy in the maximum likelihood estimation of univariate gaussian mixtures with em. Stat Prob Lett 61:373–382
Biernacki C, Celeux G, Govaert G (2003) Strategies for getting the highest likelihoods in mixture models. Guest Editors: Böhning and Seidel. Comput Stat Data Anal 41:561–575
Choi K, Bulgren WG (1968) An estimation procedure for mixtures of distributions. J R Stat Soc B 30:444–460
Clarke BR (1989) An unbiased minimum distance estimator of the proportion parameter in a mixture of two normal distributions. Stat Prob Lett 7(4):275–281
Clarke BR (2000) A review of differentiability in relation to robustness with an application to seismic data analysis. PINSA 66A:467–482
Clarke BR, Heathcote CR (1978) Comment on “Estimating mixtures of normal distributions and switching regressions” by Quandt, R.E. and Ramsey, J.B. J Am Stat Assoc 73:749–750
Clarke BR, Heathcote CR (1994) Robust estimation of \(k\)-component univariate normal mixtures. Ann Inst Stat Math 46:83–93
Clarke BR, Futshik A (2007) On the convergence of Newton’s method when estimating higher dimensional parameters. J Multivar Anal 98:916–931
Cutler A, Cordiero-Braña OI (1996) Minimum Hellinger distance estimation for finite mixture models. J Am Stat Assoc 91:1716–1723
Dempster AP, Laird NM, Rubin DP (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J R Stat Soc B 39:1–38
Depraetere N, Vandebroek M (2014) Order selection in finite mixtures of linear regressions. Stat Pap 55:871–911
Fisher RA (1947) The analysis of covariance method for the relation between a part and the whole. Biometrics 3:65–68
Fryer JG, Robertson CA (1972) A comparison of some methods for estimating mixed normal distributions. Biometrika 59:639–648
Hasselman B (2013) nleqslv: solve systems of non linear equations. R package version 2.0
Huber PJ, Ronchetti EM (2009) Robust statistics, 2nd edn. Wiley, Hoboken
Klar B, Meintanis SG (2005) Tests for normal mixtures based on the empirical characteristic function. Comput Stat Data Anal 49:227–242
Lee SX, McLachlan GJ (2013) On mixtures of skew normal and skew t-distributions. Adv Data Anal Classif 7(3):241–266
Lee SX, McLachlan GJ (2014) EMMIXuskew: an R package for fitting mixtures of multivariate skew t-distributions via the EM algorithm. J Stat Soft 55(12):1–22
Macdonald PDM (1971) Comment on a paper by Choi, K. and Bulgren, W.G. J R Stat Soc B 33:326–329
McLachlan GJ, Krishnan T (2008) The EM algorithm and extensions, 2nd edn. Wiley, New York
McLachlan GJ, Peel D (2000) Finite mixture models. Wiley, New York
McLachlan GJ, Peel D, Basford KE, Adams P (1999) The EMMIX software for the fitting of mixtures of normal and t-components. J Stat Softw 4(2):1–14
Pearson K (1894) Contributions to the mathematical theory of evolution. Philos Trans R Soc Lond A 185(1887–1895):71–110
Peel D, McLachlan GJ (2000) Robust mixture modelling using the t distribution. Stat Comput 10:339–348
Quandt RE, Ramsey JB (1978) Estimating mixtures of normal distributions and switching regressions. J Am Stat Assoc 73:730–738
R Development Core Team (2014) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0. http://www.r-project.org/
Redner RA, Walker HF (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev 26:195–239
Seidel W, Mosler K, Alker M (2000) A cautionary note on likelihood ratio tests in mixture models. Ann Inst Stat Math 52:481–487
Seidel W, Ševčíková H (2004) Types of likelihood maxima in mixture models and their implication on the performance of tests. Ann Inst Stat Math 56:631–654
Tan WY, Chang WC (1972) Some comparisons of the method of moments and the method of maximum likelihood in estimating parameters of a mixture of two normal densities. J Am Stat Assoc 67:702–708
Titterington DM, Smith AFM, Makov UE (1985) Statistical analysis of finite mixture distributions. Wiley, New York
Wang K, McLachlan GJ, Ng A, Peel D (2009) EMMIX-skew EM algorithm for mixture of multivariate skew normal/t distributions. EMMIX was originally written in Fortran by David Peel, R package version 1.0.20. http://www.maths.uq.edu.au/gjm/mix_soft/EMMIX-skew
Woodward WA, Parr WC, Schucany WR, Lindsey H (1984) A comparison of minimum distance and maximum likelihood estimation of a mixture proportion. J Am Stat Assoc 79:590–598
Wu CFJ (1983) On the convergence properties of the EM algorithm. Ann Stat 11:95–103
Acknowledgments
The authors are indebted to Emeritus Professor C.R. Heathcote, now retired, for his pioneering work on minimum distance estimators. Views expressed in this paper are those of the authors and do not necessarily represent those of the Australian Bureau of Statistics. The authors also acknowledge the helpful suggestions on presentation afforded by two anonymous referees that led to an improved paper.
Author information
Authors and Affiliations
Corresponding author
Appendix: Averages and mean squared errors of estimates
Appendix: Averages and mean squared errors of estimates
For completeness we include here averages and mean squared errors for individual parameters for parametric models (1)–(7) (Tables 10 and 11) and models (8)–(12) (Tables 12 and 13) for the \(L_2\) method and the EM algorithm for the MLE obtained using mixtools, that is EM_N.
Rights and permissions
About this article
Cite this article
Clarke, B.R., Davidson, T. & Hammarstrand, R. A comparison of the \(L_2\) minimum distance estimator and the EM-algorithm when fitting \({\varvec{{k}}}\)-component univariate normal mixtures. Stat Papers 58, 1247–1266 (2017). https://doi.org/10.1007/s00362-016-0747-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00362-016-0747-x