Abstract
In this paper we present an approximation to the mutual information between a single variable and a set of variables. The main aim of our approach is to reduce the amount of sufficient statistics (i.e. frequency counts) required to calculate the mutual information. To do so, we use the chain rule and assume different independence statements between variables. We will use our approximation to calculate the MDL of a given Bayesian network. We will show that our approximated approach to the MDL measure is score equivalent and we will use it in order to learn Bayesian networks from data. We will experimentally see that learning algorithms that use our approach obtain high quality Bayesian networks. We also note that our approach can be used in any information based measures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (2000) 4--37
Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning. Datamining, inference and prediction. Springer Series in Statistics. Springer (2001)
Roure, J.: Incremental augmented naive bayes classifiers. In: Proceedings of the sixteenth European Conference of Artificial Intelligence (ECAI 2004), IOS Press (2004)
Hulten, G., Domingos, P.: Mining complex models from arbitrarily large databases in constant time. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM Press (2002) 525--531
Meek, C., Thiesson, B., Heckerman, D.: The learning-curve sampling method applied to model-based clustering. Journal of Machine Learning Research (2002)
Buntine, W.: Theory refinement on Bayesian networks. In D’Ambrosio, B., Smets, P., Bonisone, P., eds.: Proceedings of the 7th UAI. (1991)
Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Proceedings of the Twelveth Conference on Uncertainty in Artificial Intelligence. (1996)
Lam, W., Bacchus, F.: Learning Bayesian belief networks. an approach based on the MDL principle. Computational Intelligence 10 (1994) 269--293
Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks form data. Machine Learning 9 (1992) 309--347
Chickering, D.M.: A transformational characterization of equivalent Bayesian networks. In Besnard, P., Hanks, S., eds.: Conference on Uncertainty in Artificial Intelligence, Morgan-Kaufman (1995) 87--98
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley series in telecommunications. John Wiley & Sons, Inc. (1991)
McKay, D.J.: Information theory, inference and learning algorithms. http://wol.ra.phy.cam.ac.uk/mackay/itprnn/book.ps.gz (1999)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann (1988)
Murphy, P., Aha, D.: UCI repository of Machine Learning databases. http://www.ics.uci.edu/ mlearn/MLRepository.html. (1994) Irvine, CA: University of California, Department of Information and Computer Science.
Castelo, R., Kočka, T.: On inclusion-driven learning of Bayesian networks. Journal of Machine Learning Research (2003)
Kohavi, R., John, G., Long, R., Manley, D., Pfleger, K.: MLC++: A Machine Learning library in c++. In: Proceedings of the Sixth International Conference on Tools with Artificial Intelligence, IEEE Computer Society Press (1994) 740--743
Tian, J.: A branch-and-bound algorithm for MDL learning bayesian networks. In: Proceedings of Uncertainty in Artificial Intelligence, Morgan Kauffman (2000) 580--588
Friedman, N., Getoor, L.: Efficient learning using constrained sufficient statistics. In: International Workshop on Artificial Intelligence. (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Alcobé, J.R. (2007). Learning Bayesian Networks with an Approximated MDL Score. In: Lucas, P., Gámez, J.A., Salmerón, A. (eds) Advances in Probabilistic Graphical Models. Studies in Fuzziness and Soft Computing, vol 213. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68996-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-68996-6_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68994-2
Online ISBN: 978-3-540-68996-6
eBook Packages: EngineeringEngineering (R0)