Abstract
Rule extraction is an important method seeking to understand how neural networks are able to solve problems. In order for rule extraction to be comprehensible, good knowledge representations should be used. So called M of N rules are a compact way of representing knowledge that has a strong intuitive connection to the structure of neural networks. M of N rules have been used in the past in the context of supervised models but not unsupervised models. Here we present a novel extension of a previous rule extraction algorithm for RBMs that allows us to quickly extract accurate M of N rules. The results are compared on simple datasets showing that M of N extraction has the potential to be an effective method for the knowledge representation of RBMs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Son, T., d’Avila Garcez, A.: Deep logic network: inserting and extracting knowledge from deep belief networks. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–13 (2016)
Towell, G.G., Shavlik, J.W.: The extraction of refined rules from knowledge-based neural networks. Mach. Learn. 13(1), 71–101 (1993)
Craven, M.: Extracting comprehensible models from trained neural networks. Ph.D. thesis, University of Wisconsin-Madison (1996)
Towell, G.G.: Symbolic knowledge and neural networks: insertion, refinement and extraction. Ph.D. thesis, University of Wisconsin - Madison (1991)
Murphy, P., Pazzani, M.: ID2-of-3: constructive induction of M-of -N concepts for discriminators in decision trees. In: Machine Learning: Proceedings of the Eighth International Workshop (1991)
Setiono, R.: Extracting M of N rules from trained neural networks. IEEE Trans. Neural Netw. Learn. Syst. 11(2), 512–519 (2000)
Pinkas, G.: Reasoning, connectionist nonmonotonicity and learning in networks that capture propositional knowledge. Artif. Intell. 77, 203–247 (1995)
Liu, H., Tan, S.: X2R: a fast rule generator. In: IEEE International Conference on Systems, Man and Cybernetics, Intelligent Systems for the 21st Century (1995)
Percy, C., d’Avila Garcez, A., Dragic̆ević, S. França M., Slabaugh, G., Weyde, T.: The need for knowledg extraction: understanding harmful gambling behaviour with neural networks. In: 22nd European Conference on Artificial Intelligence, pp. 974–981 (2016)
d’Avila Garcez, A., Lamb, L., Gabbay, D.: Neural-Symbolic Cognitive Reasoning. Springer, New York (2009)
Kautz, W.: The realization of symmetric switching functions with linear-input logical elements. IRE Trans. Electron. Comput. EC–10(3), 371–378 (1961)
Hinton, G.E., Osindero, S., Teh, T.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Smolensky, P.: Information proessing in dynamical systems: foundations of harmony theory. In: Parallel Distributed Processing, vol. 1, pp. 194–281. Foundations. MIT Press, Cambridge (1986)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Odense, S., d’Avila Garcez, A. (2017). Extracting M of N Rules from Restricted Boltzmann Machines. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10614. Springer, Cham. https://doi.org/10.1007/978-3-319-68612-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-68612-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68611-0
Online ISBN: 978-3-319-68612-7
eBook Packages: Computer ScienceComputer Science (R0)