Skip to main content
SpringerLink
Log in

Maximum Likelihood Estimates for Gaussian Mixtures Are Transcendental

  • Conference paper
  • First Online:
Mathematical Aspects of Computer and Information Sciences (MACIS 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9582))

Abstract

Gaussian mixture models are central to classical statistics, widely used in the information sciences, and have a rich mathematical structure. We examine their maximum likelihood estimates through the lens of algebraic statistics. The MLE is not an algebraic function of the data, so there is no notion of ML degree for these models. The critical points of the likelihood function are transcendental, and there is no bound on their number, even for mixtures of two univariate Gaussians.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Améndola, C., Faugère, J.-C., Sturmfels, B.: Moment varieties of Gaussian mixtures. J. Algebraic Stat. arXiv:1510.04654

  2. Baker, A.: Transcendental Number Theory. Cambridge University Press, London (1975)

    Book  MATH  Google Scholar 

  3. Belkin, M., Sinha, K.: Polynomial learning of distribution families. SIAM J. Comput. 44(4), 889–911 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, New York (2006)

    MATH  Google Scholar 

  5. Buot, M., Hoşten, S., Richards, D.: Counting and locating the solutions of polynomial systems of maximum likelihood equations. II. The Behrens-Fisher problem. Stat. Sin. 17(4), 1343–1354 (2007)

    MathSciNet  MATH  Google Scholar 

  6. Chang, E.-C., Choi, S.W., Kwon, D., Park, H., Yap, C.: Shortest paths for disc obstacles is computable. Int. J. Comput. Geom. Appl. 16, 567–590 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Choi, S.W., Pae, S., Park, H., Yap, C.: Decidability of collision between a helical motion and an algebraic motion. In: Hanrot, G., Zimmermann, P. (eds.) 7th Conference on Real Numbers and Computers, pp. 69–82. LORIA, Nancy (2006)

    Google Scholar 

  8. Drton, M., Sturmfels, B., Sullivant, S.: Lectures on Algebraic Statistics. Oberwolfach Seminars, vol. 39. Birkhäuser, Basel (2009)

    Book  MATH  Google Scholar 

  9. Fraley, C., Raftery, A.E.: Enhanced model-based clustering, density estimation, and discriminant analysis software: MCLUST. J. Classif. 20, 263–286 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  10. Ge, R., Huang, Q., Kakade, S.: Learning mixtures of Gaussians in high dimensions. In: STOC 2015, Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pp. 761–770 (2015)

    Google Scholar 

  11. Gelfond, A.O.: Transcendental and Algebraic Numbers. Translated by Leo F. Boron, Dover Publications, New York (1960)

    Google Scholar 

  12. Gross, E., Drton, M., Petrović, S.: Maximum likelihood degree of variance component models. Electron. J. Stat. 6, 993–1016 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  13. Huh, J., Sturmfels, B.: Likelihood geometry. In: Conca, A., et al. (eds.) Combinatorial Algebraic Geometry. Lecture Notes in Math., vol. 2108, pp. 63–117. Springer, Heidelberg (2014)

    Google Scholar 

  14. Moitra, A., Valiant, G.: Settling the polynomial learnability of mixtures of Gaussians. In: IEEE 51st Annual Symposium on Foundations of Computer Science, pp. 93–102 (2010)

    Google Scholar 

  15. Pearson, K.: Contributions to the mathematical theory of evolution. Philos. Trans. R. Soc. Lond. A 185, 71–110 (1894)

    Article  MATH  Google Scholar 

  16. Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26, 195–239 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  17. Reeds, J.A.: Asymptotic number of roots of Cauchy location likelihood equations. Ann. Statist. 13(2), 775–784 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  18. Srebro, N.: Are there local maxima in the infinite-sample likelihood of Gaussian mixture estimation? In: Bshouty, N.H., Gentile, C. (eds.) COLT. LNCS (LNAI), vol. 4539, pp. 628–629. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  19. Sturmfels, B., Uhler, C.: Multivariate Gaussian, semidefinite matrix completion, and convex algebraic geometry. Ann. Inst. Statist. Math. 62(4), 603–638 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  20. Teicher, H.: Identifiability of finite mixtures. Ann. Math. Stat. 34, 1265–1269 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  21. Watanabe, S.: Algebraic Geometry and Statistical Learning Theory. Monographs on Applied and Computational Mathematics, vol. 25. Cambridge University Press, Cambridge (2009)

    Book  MATH  Google Scholar 

  22. Watanabe, S., Yamazaki, K., Aoyagi, M.: Kullback information of normal mixture is not an analytic function. IEICE Technical report, NC2004-50 (2004)

    Google Scholar 

Download references

Acknowledgements

CA and BS were supported by the Einstein Foundation Berlin. MD and BS also thank the US National Science Foundation (DMS-1305154 and DMS-1419018).

Author information

Authors and Affiliations

  1. Technische Universität, Berlin, Germany

    Carlos Améndola & Bernd Sturmfels

  2. University of Washington, Seattle, USA

    Mathias Drton

  3. University of California, Berkeley, USA

    Bernd Sturmfels

Authors
  1. Carlos Améndola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathias Drton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernd Sturmfels
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Améndola .

Editor information

Editors and Affiliations

  1. Wilfrid Laurier University, Waterloo, Ontario, Canada

    Ilias S. Kotsireas

  2. Hamburg University of Technology, Hamburg, Germany

    Siegfried M. Rump

  3. New York University, New York, New York, USA

    Chee K. Yap

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Améndola, C., Drton, M., Sturmfels, B. (2016). Maximum Likelihood Estimates for Gaussian Mixtures Are Transcendental. In: Kotsireas, I., Rump, S., Yap, C. (eds) Mathematical Aspects of Computer and Information Sciences. MACIS 2015. Lecture Notes in Computer Science(), vol 9582. Springer, Cham. https://doi.org/10.1007/978-3-319-32859-1_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32859-1_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32858-4

  • Online ISBN: 978-3-319-32859-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation