Skip to main content
SpringerLink
Log in

A Bayesian Information Criterion for Unsupervised Learning Based on an Objective Prior

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2019 (ICCSA 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11619))

Included in the following conference series:

Abstract

Data processing techniques, such as mathematical formulas, statistical methods and machine learning algorithms, require a set of tools for evaluating knowledge extracted from data. In unsupervised learning it is impossible to use referential or predictive estimation. Therefore, the only reliable way to evaluate results of unsupervised learning is information estimation. Unfortunately, information estimation suffer from underfitting and overfitting. We propose a new method for evaluating unsupervised learning results, which is based on the Bayesian criterion for optimal decision and an objective prior probability distribution of partitions. We illustrate the proposed method application on Fisher’s iris data set by comparing original label distribution with results of clustering with different numbers of clusters. We show the method prevents underfitting and overfitting and verify it by comparing the recommended value with posterior distribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Piironen, J., Vehtari, A.: Comparison of Bayesian predictive methods for model selection. Stat. Comput. 27(3), 711–735 (2017). https://doi.org/10.1007/s11222-016-9649-y

    Article  MATH  MathSciNet  Google Scholar 

  2. Lever, J., Krzywinski, M., Altman, N.: Points of significance: model selection and overfitting. Nat. Methods 13(9), 703–704 (2016)

    Article  Google Scholar 

  3. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)

    Article  Google Scholar 

  4. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)

    Book  Google Scholar 

  5. Simpson, D., Rue, H., Riebler, A., Martins, T.G., Sørbye, S.H.: Penalising model component complexity: a principled, practical approach to constructing priors. Stat. Sci. 32(1), 1–28 (2017)

    Article  MathSciNet  Google Scholar 

  6. Mattingly, H.H., Transtrum, M.K., Abbott, M.C., Machta, B.B.: Maximizing the information learned from finite data selects a simple model. Proc. Nat. Acad. Sci. U.S.A. 115(8), 1760–1765 (2018)

    Article  Google Scholar 

  7. Palmieri, F.A.N., Ciuonzo, D.: Objective priors from maximum entropy in data classification. Inf. Fusion 14(2), 186–198 (2013)

    Article  Google Scholar 

  8. Sørbye, S.H., Rue, H.: Penalised complexity priors for stationary autoregressive processes. J. Time Ser. Anal. 38(6), 923–935 (2017)

    Article  MathSciNet  Google Scholar 

  9. Quinlan, J.R.: Induction of decision trees. Mach. learn. 1(1), 81–106 (1986)

    Google Scholar 

  10. Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010)

    MATH  MathSciNet  Google Scholar 

  11. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  Google Scholar 

  12. Berger, J.O.: Statistical Decision Theory and Bayesian Analysis. Springer Series in Statistics, 2nd edn. Springer, New York (1985). https://doi.org/10.1007/978-1-4757-4286-2

    Book  MATH  Google Scholar 

  13. Ward Jr., J.H.: Hierarchical grouping to optimize an objective function. J. Am. Stat. Assoc. 58(301), 236–244 (1963)

    Article  MathSciNet  Google Scholar 

  14. Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007)

    Article  MathSciNet  Google Scholar 

  15. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  16. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, pp. 226–231, Portland (1996)

    Google Scholar 

Download references

Acknowledgments

The research was funded by RFBR and CITMA according to the research project №18-57-34001 and was funded by RFBR according to the research project №19-07-00784

Author information

Authors and Affiliations

  1. ITMO University, St. Petersburg, Russia

    Ildar Baimuratov, Nataly Zhukova & Nguyen Than

  2. St. Petersburg State Electrotechnical University, St. Petersburg, Russia

    Yulia Shichkina

  3. St. Petersburg State University, St. Petersburg, Russia

    Elena Stankova

  4. St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia

    Nataly Zhukova

Authors
  1. Ildar Baimuratov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yulia Shichkina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elena Stankova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nataly Zhukova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nguyen Than
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yulia Shichkina .

Editor information

Editors and Affiliations

  1. Covenant University, Ota, Nigeria

    Sanjay Misra

  2. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  3. University of Basilicata, Potenza, Italy

    Beniamino Murgante

  4. Saint Petersburg State University, Saint Petersburg, Russia

    Elena Stankova

  5. Saint Petersburg State University, Saint Petersburg, Russia

    Vladimir Korkhov

  6. Polytechnic University of Bari, Bari, Italy

    Carmelo Torre

  7. University of Minho, Braga, Portugal

    Ana Maria A.C. Rocha

  8. Monash University, Clayton, VIC, Australia

    David Taniar

  9. Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  10. Polytechnic University of Bari, Bari, Italy

    Eufemia Tarantino

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baimuratov, I., Shichkina, Y., Stankova, E., Zhukova, N., Than, N. (2019). A Bayesian Information Criterion for Unsupervised Learning Based on an Objective Prior. In: Misra, S., et al. Computational Science and Its Applications – ICCSA 2019. ICCSA 2019. Lecture Notes in Computer Science(), vol 11619. Springer, Cham. https://doi.org/10.1007/978-3-030-24289-3_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-24289-3_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-24288-6

  • Online ISBN: 978-3-030-24289-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation