Skip to main content
SpringerLink
Log in

Learning Theories Using Estimation Distribution Algorithms and (Reduced) Bottom Clauses

  • Conference paper
Inductive Logic Programming (ILP 2011)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7207))

Included in the following conference series:

Abstract

Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability they were applied (to some extent) to Inductive Logic Programming (ILP).  Although Estimation of Distribution Algorithms (EDAs) generally perform better than standard GAs, they have not been applied to ILP.  This work presents EDA-ILP, an ILP system based on EDA and inverse entailment, and also its extension, the REDA-ILP, which employs the Reduce algorithm in bottom clauses to considerably reduce the search space. Experiments in real-world datasets showed that both systems were successfully compared to Aleph and GA-ILP (another variant of EDA-ILP created replacing the EDA by a standard GA). EDA-ILP was also successfully compared to Progol-QG/GA (and its other variants) in phase transition benchmarks. Additionally, we found that REDA-ILP usually obtains simpler theories than EDA-ILP, more efficiently and with equivalent accuracies. These results show that EDAs provide a good base for stochastic search in ILP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Pitangui, C., Zaverucha, G.: Inductive Logic Programming Through Estimation Distribution Algorithm. In: Proceedings of IEEE Congress of Evolutionary Computation (CEC 2011), New Orleans, LA, EUA, pp. 54–61 (2011) 978-1-4244-7834-7

    Google Scholar 

  2. Muggleton, S., De Raedt, L.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19(20) (1994)

    Google Scholar 

  3. Mühlenbein, H., Paaß, G.: From Recombination of Genes to the Estimation of Distributions I. Binary Parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  4. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)

    Google Scholar 

  5. Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Carnegie Mellon University, Pittsburgh (1994); Technical Report: CMU-CS-94-163

    Google Scholar 

  6. Holland, J.: Adaptation in natural and artificial systems. MIT Press, Cambridge (1975)

    Google Scholar 

  7. Muggleton, S.H., Feng, C.: Efficient induction of logic programs. In: Proceedings of the First Conference on Algorithmic Learning Theory, pp. 368–381. Ohmsha, Tokyo (1990)

    Google Scholar 

  8. Srinivasan, A.: The Aleph Manual, www.comlab.ox.ac.uk/activities/machinelearning/Aleph/ (last access September 29, 2011)

  9. Alphonse, E., Rouveirol, C.: Lazy propositionalisation for Relational Learning. In: 14th European Conference on Artificial Intelligence (ECAI 2000), pp. 256–260. IOS Press (2000)

    Google Scholar 

  10. Muggleton, S., Tamaddoni-Nezhad, A.: QG/GA: A stochastic search approach for Progol. Machine Learning 70(2-3), 123–133 (2007), doi:10.1007/s10994-007-5029-3

    Google Scholar 

  11. Muggleton, S.: Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming 13(3-4), 245–286 (1995)

    Google Scholar 

  12. Oliphant, L., Shavlik, J.: Using Bayesian Networks to Direct Stochastic Search in Inductive Logic Programming. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 191–199. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  13. Srinivasan, A., King, R.D.S.H., Muggleton, S., Sternberg, M.: Carcinogenesis Predictions using ILP. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS (LNAI), vol. 1297, pp. 273–287. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  14. King, R.D., Srinivasan, A., Sternberg, M.J.E.: Relating chemical activity to structure: an examination of ILP successes. New Gen. Comp. 13, 411–433 (1995)

    Article  Google Scholar 

  15. Nadeau, C., Bengio, Y.: Inference for the Generalization Error. Machine Learning 52(3), 239–281 (2003)

    Article  MATH  Google Scholar 

  16. Huynh, T., Mooney, R.: Discriminative Structure and Parameter Learning for Markov Logic Networks. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), Helsinki, Finland, pp. 416–423 (2008)

    Google Scholar 

  17. Muggleton, S.H., Santos, J.C.A., Tamaddoni-Nezhad, A.: TopLog: ILP Using a Logic Program Declarative Bias. In: Garcia de la Banda, M., Pontelli, E. (eds.) ICLP 2008. LNCS, vol. 5366, pp. 687–692. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  18. Bratko, I.: Refining Complete Hypotheses in ILP. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 44–55. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  19. Paes, A., Zaverucha, G., Santos Costa, V.: Revising First-Order Logic Theories from Examples Through Stochastic Local Search. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 200–210. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  20. Srinivasan, A.: A study of two probabilistic methods for searching large spaces with ILP(Technical Report PRG-TR-16-00). Oxford University Computing Laboratory, Oxford (2000)

    Google Scholar 

  21. Paes, A., Železný, F., Zaverucha, G., Page, D.L., Srinivasan, A.: ILP Through Propositionalization and Stochastic k-Term DNF Learning. In: Muggleton, S.H., Otero, R., Tamaddoni-Nezhad, A. (eds.) ILP 2006. LNCS (LNAI), vol. 4455, pp. 379–393. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  22. Tamaddoni-Nezhad, A., Muggleton, S.H.: Searching the Subsumption Lattice by a Genetic Algorithm. In: Cussens, J., Frisch, A.M. (eds.) ILP 2000. LNCS (LNAI), vol. 1866, pp. 243–252. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  23. Pelikan, M.: Hierarchical Bayesian Optimization Algorithm Toward a New Generation of Evolutionary Algorithms, 1st edn. STUDFUZZ, vol. 170. Springer (2005)

    Google Scholar 

  24. Železný, F., Srinivasan, A., Page, D.: Lattice-Search Runtime Distributions May Be Heavy-Tailed. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 333–345. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  25. Goadrich, M., Oliphant, L., Shavlik, J.: Gleaner: Creating Ensembles of First-Order Clauses to Improve Recall-Precision Curves. Machine Learning 64(1-3), 231–261 (2006)

    Article  MATH  Google Scholar 

  26. Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational learning as search in a critical region. Journal of Machine Learning Research 4, 431–463 (2003)

    MathSciNet  Google Scholar 

  27. Alphonse, E., Osmani, A.: On the connection between the phase transition of the covering test and the learning success rate in ILP. Machine Learning Journal 70(2-3), 135–150 (2008)

    Article  Google Scholar 

  28. Henrion, M.: Propagating Uncertainty in Bayesian Networks by Probabilistic Logic Sampling. In: Lemmer, J.F., Kanal, L.N. (eds.) Uncertainty in Artificial Intelligence, vol. 2, pp. 149–163. North Holland (1988)

    Google Scholar 

  29. Pitangui, C., Zaverucha, G.: Genetic local search for rule learning. In: Genetic And Evolutionary Computation Conference (GECCO) Atlanta, GA, USA, pp. 1427–1428 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PESC - COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ, Brazil

    Cristiano Grijó Pitangui & Gerson Zaverucha

Authors
  1. Cristiano Grijó Pitangui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerson Zaverucha
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, Imperial College London, 180 Queen’s Gate, SW7 2AZ, London, UK

    Stephen H. Muggleton  & Alireza Tamaddoni-Nezhad  & 

  2. Dipartimento di Informatica, Università degli Studi di Bari “Aldo Moro”, Via E. Orabona, 4, 70125, Bari, Italy

    Francesca A. Lisi

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pitangui, C.G., Zaverucha, G. (2012). Learning Theories Using Estimation Distribution Algorithms and (Reduced) Bottom Clauses. In: Muggleton, S.H., Tamaddoni-Nezhad, A., Lisi, F.A. (eds) Inductive Logic Programming. ILP 2011. Lecture Notes in Computer Science(), vol 7207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31951-8_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-31951-8_25

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-31950-1

  • Online ISBN: 978-3-642-31951-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation