Skip to main content
SpringerLink
Log in

Widened Learning of Bayesian Network Classifiers

  • Conference paper
  • First Online:
Advances in Intelligent Data Analysis XV (IDA 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9897))

Included in the following conference series:

Abstract

We demonstrate the application of Widening to learning performant Bayesian Networks for use as classifiers. Widening is a framework for utilizing parallel resources and diversity to find models in a hypothesis space that are potentially better than those of a standard greedy algorithm. This work demonstrates that widened learning of Bayesian Networks, using the Frobenius Norm of the networks’ graph Laplacian matrices as a distance measure, can create Bayesian networks that are better classifiers than those generated by popular Bayesian Network algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We freely mix the use of “solution space” and “hypothesis space” throughout this paper, referring essentially to the same space, but drawing attention to whether it is the evaluation of the hypothesis or the hypothesis itself that is important.

  2. 2.

    In this application, it would be correctly termed “l-dispersion-min-sum,” but the notation is written here as “p” to be consistent with the literature.

  3. 3.

    http://archive.ics.uci.edu/ml/.

  4. 4.

    http://www.csc.liv.ac.uk/~frans/KDD/Software/LUCS_KDD_DN/.

  5. 5.

    http://www.bnlearn.com/.

References

  1. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19(6), 716–723 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  2. Akbar, Z., Ivanova, V.N., Berthold, M.R.: Parallel data mining revisited. better, not faster. In: Hollmén, J., Klawonn, F., Tucker, A. (eds.) IDA 2012. LNCS, vol. 7619, pp. 23–34. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34156-4_4

    Chapter  Google Scholar 

  3. Berthold, M.R., Cebron, N., Dill, F., Gabriel, T.R., Kötter, T., Ohl, P., Sieb, C., Thiel, K., Wiswedel, B.: KNIME: the konstanz information miner. In: Preisach, C., Burkhardt, H., Schmidt-Thieme, L., Decker, R. (eds.) Data Analysis, Machine Learning and Applications. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 319–326. Springer, Heidelberg (2007)

    Google Scholar 

  4. Bielza, C., Larranaga, P.: Discrete Bayesian network classifiers: a survey. ACM Comput. Surv. (CSUR) 47(1), 5 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  5. Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 52–60. Morgan Kaufmann Publishers Inc., Los Angeles (1991)

    Google Scholar 

  6. Carvalho, A.M.: Scoring functions for learning Bayesian networks. Technical report INESC-ID Technical report 54/2009, Instituto superior Téchnico, Technical University of Lisboa, April 2009

    Google Scholar 

  7. Cheng, J., Bell, D.A., Liu, W.: An algorithm for Bayesian belief network construction from data. In: Proceedings of AI & STAT 1997, pp. 83–90 (1997)

    Google Scholar 

  8. Chickering, D.M.: Optimal structure identification with greedy search. J. Mach. Learn. Res. 3, 507–554 (2002)

    MathSciNet  MATH  Google Scholar 

  9. Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)

    MATH  Google Scholar 

  10. De Campos, C.P., Zeng, Z., Ji, Q.: Structure learning of Bayesian networks using constraints. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 113–120. ACM (2009)

    Google Scholar 

  11. Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, New York (1973)

    MATH  Google Scholar 

  12. Erkut, E.: The discrete p-dispersion problem. Eur. J. Oper. Res. 46(1), 48–60 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  13. Fillbrunn, A., Berthold, M.R.: Diversity-driven widening of hierarchical agglomerative clustering. In: Fromont, E., Bie, T., Leeuwen, M. (eds.) IDA 2015. LNCS, vol. 9385, pp. 84–94. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24465-5_8

    Chapter  Google Scholar 

  14. Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2–3), 131–163 (1997)

    Article  MATH  Google Scholar 

  15. Golub, G.H., van Loan, C.F.: Matrix Computations, 4th edn. The Johns Hobpkins University Press, Baltimore (2013)

    MATH  Google Scholar 

  16. Hamming, R.W.: Error detecting and error correcting codes. Bell Syst. Tech. J. 29(2), 147–160 (1950)

    Article  MathSciNet  Google Scholar 

  17. Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)

    MATH  Google Scholar 

  18. Ivanova, V.N., Berthold, M.R.: Diversity-driven widening. In: Tucker, A., Höppner, F., Siebes, A., Swift, S. (eds.) IDA 2013. LNCS, vol. 8207, pp. 223–236. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41398-8_20

    Chapter  Google Scholar 

  19. Koski, T.J., Noble, J.M.: A review of Bayesian networks and structure learning. Math. Applicanda 40(1), 53–103 (2012)

    MathSciNet  MATH  Google Scholar 

  20. Larrañaga, P., Karshenas, H., Bielza, C., Santana, R.: A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf. Sci. 233, 109–125 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Lichman, M.: UCI Machine Learning Repository (2013)

    Google Scholar 

  22. Lowerre, B.T.: The HARPY speech recognition system. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA, USA (1976)

    Google Scholar 

  23. Maron, M.E., Kuhns, J.L.: On relevance, probabilistic indexing and information retrieval. J. ACM (JACM) 7(3), 216–244 (1960)

    Article  Google Scholar 

  24. Meinl, T.: Maximum-score diversity selection. Ph.D. thesis, University of Konstanz, July 2010

    Google Scholar 

  25. Nielsen, J.D., Kočka, T., Peña, J.M.: On local optima in learning Bayesian networks. In: Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, pp. 435–442. Morgan Kaufmann Publishers Inc., San Francisco (2003)

    Google Scholar 

  26. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)

    MATH  Google Scholar 

  27. Pernkopf, F.: Bayesian network classifiers versus k-NN classifier using sequential feature selection. In: AAAI, pp. 360–365 (2004)

    Google Scholar 

  28. Robinson, R.W.: Counting unlabeled acyclic digraphs. In: Little, C.H.C. (ed.) Combinatorial Mathematics V. LNM, vol. 622, pp. 28–43. Springer, Heidelberg (1977)

    Chapter  Google Scholar 

  29. Sampson, O., Berthold, M.R.: Widened KRIMP: better performance through diverse parallelism. In: Blockeel, H., Leeuwen, M., Vinciotti, V. (eds.) IDA 2014. LNCS, vol. 8819, pp. 276–285. Springer, Heidelberg (2014). doi:10.1007/978-3-319-12571-8_24

    Google Scholar 

  30. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  31. Sierra, B., Larrañaga, P.: Predicting the survival in malignant skin melanoma using Bayesian networks. an empirical comparison between different approaches. Artif. Intell. Med. 14(1–2), 215–230 (1998)

    Article  Google Scholar 

  32. Sprites, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. MIT Press, Cambridge (1993)

    Book  MATH  Google Scholar 

  33. Jiang, S., Zhang, H.: Full Bayesian network classifiers. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 897–904. ACM (2006)

    Google Scholar 

  34. Suzuki, J.: A construction of Bayesian networks from databases based on an MDL principle. In: Proceedings of the Ninth International Conference on Uncertainty in Artificial Intelligence, pp. 266–273. Morgan Kaufmann Publishers Inc., San Francisco (1993)

    Google Scholar 

  35. Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65(1), 31–78 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chair for Bioinformatics and Information Mining, Department of Computer and Information Science, University of Konstanz, Konstanz, Germany

    Oliver R. Sampson & Michael R. Berthold

Authors
  1. Oliver R. Sampson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael R. Berthold
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oliver R. Sampson .

Editor information

Editors and Affiliations

  1. Stockholm University , Stockholm, Sweden

    Henrik Boström

  2. Leiden University , Leiden, The Netherlands

    Arno Knobbe

  3. University of Porto , Porto, Portugal

    Carlos Soares

  4. Stockholm University , Stockholm, Sweden

    Panagiotis Papapetrou

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Sampson, O.R., Berthold, M.R. (2016). Widened Learning of Bayesian Network Classifiers. In: Boström, H., Knobbe, A., Soares, C., Papapetrou, P. (eds) Advances in Intelligent Data Analysis XV. IDA 2016. Lecture Notes in Computer Science(), vol 9897. Springer, Cham. https://doi.org/10.1007/978-3-319-46349-0_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46349-0_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46348-3

  • Online ISBN: 978-3-319-46349-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation