Skip to main content
SpringerLink
Log in

Learning Probabilistic Logic Programs over Continuous Data

  • Conference paper
  • First Online:
Inductive Logic Programming (ILP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11770))

Included in the following conference series:

Abstract

The field of statistical relational learning aims at unifying logic and probability to reason and learn from data. Perhaps the most successful paradigm in the field is probabilistic logic programming (PLP): the enabling of stochastic primitives in logic programming. While many systems offer inference capabilities, the more significant challenge is that of learning meaningful and interpretable symbolic representations from data. In that regard, inductive logic programming and related techniques have paved much of the way for the last few decades, but a major limitation of this exciting landscape is that only discrete features and distributions are handled. Many disciplines express phenomena in terms of continuous models.

In this paper, we propose a new computational framework for inducing probabilistic logic programs over continuous and mixed discrete-continuous data. Most significantly, we show how to learn these programs while making no assumption about the true underlying density. Our experiments show the promise of the proposed framework.

This work is partly supported by the EPSRC grant Towards Explainable and Robust Statistical AI: A Symbolic Approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Imagine, for example, the predicate \(\mathtt{height(X)}\) with examples such as \(\mathtt{height(60.4),\ldots ,height(91.1),\ldots ,height(124.6)}\).

  2. 2.

    That is, we may want to penalise very granular representations that are defined over a large number of intervals and polynomials of a high degree. So, we would like to minimise the loss, but prefer simpler representations over granular ones.

  3. 3.

    In an independent and recent effort, Martires et al. [30] have also considered the use of semirings to do WMI inference over propositional circuits.

References

  1. Alberti, M., Bellodi, E., Cota, G., Riguzzi, F., Zese, R.: cplint on SWISH: probabilistic logical inference with a web browser. Intell. Arti. 11(1), 47–64 (2017)

    Google Scholar 

  2. Antanas, L., Frasconi, P., Costa, F., Tuytelaars, T., De Raedt, L.: A relational kernel-based framework for hierarchical image understanding. In: Gimel’farb, G., et al. (eds.) SSPR /SPR 2012. LNCS, vol. 7626, pp. 171–180. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34166-3_19

    Chapter  Google Scholar 

  3. Baldoni, V., Berline, N., De Loera, J., Köppe, M., Vergne, M.: How to integrate a polynomial over a simplex. Math. Comput. 80(273), 297–325 (2011)

    Article  MathSciNet  Google Scholar 

  4. Baral, C., Gelfond, M., Rushton, J.N.: Probabilistic reasoning with answer sets. TPLP 9(1), 57–144 (2009)

    MathSciNet  MATH  Google Scholar 

  5. Belle, V., Van den Broeck, G., Passerini, A.: Hashing-based approximate probabilistic inference in hybrid domains. In: UAI (2015)

    Google Scholar 

  6. Belle, V., Passerini, A., Van den Broeck, G.: Probabilistic inference in hybrid domains by weighted model integration. In: Proceedings of 24th International Joint Conference on Artificial Intelligence (IJCAI), pp. 2770–2776 (2015)

    Google Scholar 

  7. Bellodi, E., Riguzzi, F.: Learning the structure of probabilistic logic programs. In: Muggleton, S.H., Tamaddoni-Nezhad, A., Lisi, F.A. (eds.) ILP 2011. LNCS (LNAI), vol. 7207, pp. 61–75. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31951-8_10

    Chapter  Google Scholar 

  8. Bellodi, E., Riguzzi, F.: Structure learning of probabilistic logic programs by searching the clause space. Theory Pract. Logic Program. 15(2), 169–212 (2015)

    Article  Google Scholar 

  9. Chavira, M., Darwiche, A.: On probabilistic inference by weighted model counting. Artif. Intell. 172(6–7), 772–799 (2008)

    Article  MathSciNet  Google Scholar 

  10. Darwiche, A., Marquis, P.: A knowledge compilation map. J. Artif. Intell. Res. 17, 229–264 (2002)

    Article  MathSciNet  Google Scholar 

  11. De Boor, C., De Boor, C., Mathématicien, E.U., De Boor, C., De Boor, C.: A practical guide to splines, vol. 27. Springer, New York (1978)

    Book  Google Scholar 

  12. De Maeyer, D., Renkens, J., Cloots, L., De Raedt, L., Marchal, K.: Phenetic: network-based interpretation of unstructured gene lists in E. coli. Mol. BioSyst. 9(7), 1594–1603 (2013)

    Article  Google Scholar 

  13. De Raedt, L., Dries, A., Thon, I., Van den Broeck, G., Verbeke, M.: Inducing probabilistic relational rules from probabilistic examples. In: Proceedings of 24th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1835–1842 (2015)

    Google Scholar 

  14. De Raedt, L., Kimmig, A.: Probabilistic (logic) programming concepts. Mach. Learn. 100(1), 5–47 (2015). https://doi.org/10.1007/s10994-015-5494-z

    Article  MathSciNet  MATH  Google Scholar 

  15. Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml

  16. Dougherty, J., Kohavi, R., Sahami, M., et al.: Supervised and unsupervised discretization of continuous features. In: Machine Learning: Proceedings of the Twelfth International Conference, vol. 12, pp. 194–202 (1995)

    Google Scholar 

  17. Džeroski, S., Cestnik, B., Petrovski, I.: Using the m-estimate in rule induction. J. Comput. Inf. Technol. 1(1), 37–46 (1993)

    Google Scholar 

  18. Fierens, D., Van den Broeck, G., Thon, I., Gutmann, B., Raedt, L.D.: Inference in probabilistic logic programs using weighted CNF’s. In: UAI, pp. 211–220 (2011)

    Google Scholar 

  19. Getoor, L., Friedman, N., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Džeroski, S., Lavrač, N. (eds.) Relational data Mining, pp. 307–335. Springer, Heidelberg (2001). https://doi.org/10.1007/978-3-662-04599-2_13

    Chapter  Google Scholar 

  20. Goodman, N.D., Mansinghka, V.K., Roy, D.M., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: Proceedings of UAI, pp. 220–229 (2008)

    Google Scholar 

  21. Gutmann, B., Jaeger, M., De Raedt, L.: Extending ProbLog with continuous distributions. In: Frasconi, P., Lisi, F.A. (eds.) ILP 2010. LNCS (LNAI), vol. 6489, pp. 76–91. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21295-6_12

    Chapter  Google Scholar 

  22. Islam, M.A., Ramakrishnan, C., Ramakrishnan, I.: Parameter learning in prism programs with continuous random variables. arXiv preprint arXiv:1203.4287 (2012)

  23. Kimmig, A., Van den Broeck, G., De Raedt, L.: An algebraic prolog for reasoning about possible worlds. In: Proceedings of the AAAI (2011). http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/view/3685

  24. Kok, S., Domingos, P.: Learning the structure of Markov logic networks. In: Proceedings of the International Conference on Machine Learning, pp. 441–448 (2005)

    Google Scholar 

  25. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  26. Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: integrating Naïve Bayes and FOIL. In: AAAI 2005, pp. 795–800 (2005)

    Google Scholar 

  27. Landwehr, N., Passerini, A., De Raedt, L., Frasconi, P., et al.: kFOIL: learning simple relational kernels. AAAI 6, 389–394 (2006)

    Google Scholar 

  28. Lauritzen, S.L., Jensen, F.: Stable local computation with conditional gaussian distributions. Stat. Comput. 11(2), 191–203 (2001)

    Article  MathSciNet  Google Scholar 

  29. López-Cruz, P.L., Bielza, C., Larrañaga, P.: Learning mixtures of polynomials of multidimensional probability densities from data using b-spline interpolation. Int. J. Approximate Reasoning 55(4), 989–1010 (2014)

    Article  MathSciNet  Google Scholar 

  30. Martires, P.Z.D., Dries, A., Raedt, L.D.: Knowledge compilation with continuous random variables and its application in hybrid probabilistic logic programming (2018). http://arxiv.org/abs/1807.00614

  31. Milch, B., Marthi, B., Russell, S.J., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: probabilistic models with unknown objects. In: Proceedings of the IJCAI, pp. 1352–1359 (2005)

    Google Scholar 

  32. Muggleton, S.: Inverse entailment and progol. New Gener. Comput. 13(3), 245–286 (1995)

    Article  Google Scholar 

  33. Murphy, K.P.: A variational approximation for Bayesian networks with discrete and continuous latent variables. In: UAI, pp. 457–466 (1999)

    Google Scholar 

  34. Nitti, D., Laet, T.D., Raedt, L.D.: A particle filter for hybrid relational domains. In: IROS, pp. 2764–2771 (2013)

    Google Scholar 

  35. Nitti, D., Ravkic, I., Davis, J., De Raedt, L.: Learning the structure of dynamic hybrid relational models. In: ECAI 2016, vol. 285, pp. 1283–1290 (2016)

    Google Scholar 

  36. Pasula, H., Marthi, B., Milch, B., Russell, S.J., Shpitser, I.: Identity uncertainty and citation matching. In: NIPS, pp. 1401–1408 (2002). http://papers.nips.cc/paper/2149-identity-uncertainty-and-citation-matching

  37. Poole, D., Bacchus, F., Kisyński, J.: Towards completely lifted search-based probabilistic inference. CoRR abs/1107.4035 (2011)

    Google Scholar 

  38. Quinlan, J.R.: Learning logical definitions from relations. Mach. Learn. 5(3), 239–266 (1990)

    Google Scholar 

  39. Raedt, L.D., Kersting, K., Natarajan, S., Poole, D.: Statistical Relational Artificial Intelligence: Logic, Probability, and Computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 10, no. 2, pp. 1–189 (2016)

    Google Scholar 

  40. Raghavan, S., Mooney, R.J., Ku, H.: Learning to read between the lines using Bayesian logic programs. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 349–358. Association for Computational Linguistics (2012)

    Google Scholar 

  41. Ravkic, I., Ramon, J., Davis, J.: Learning relational dependency networks in hybrid domains. Mach. Learn. 100(2–3), 217–254 (2015)

    Article  MathSciNet  Google Scholar 

  42. Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1), 107–136 (2006)

    Article  Google Scholar 

  43. Russell, S.: Unifying logic and probability. Commun. ACM 58(7), 88–97 (2015)

    Article  Google Scholar 

  44. Schoenmackers, S., Etzioni, O., Weld, D.S., Davis, J.: Learning first-order horn clauses from web text. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1088–1098. Association for Computational Linguistics (2010)

    Google Scholar 

  45. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MathSciNet  Google Scholar 

  46. Zong, Z.: Information-Theoretic Methods for Estimating of Complicated Probability Distributions, vol. 207. Elsevier (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Edinburgh, Edinburgh, UK

    Stefanie Speichert & Vaishak Belle

  2. Alan Turing Institute, London, UK

    Vaishak Belle

Authors
  1. Stefanie Speichert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vaishak Belle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefanie Speichert .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of York, Heslington, UK

    Dimitar Kazakov

  2. Department of Computer Science, University of York, Heslington, UK

    Can Erten

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Speichert, S., Belle, V. (2020). Learning Probabilistic Logic Programs over Continuous Data. In: Kazakov, D., Erten, C. (eds) Inductive Logic Programming. ILP 2019. Lecture Notes in Computer Science(), vol 11770. Springer, Cham. https://doi.org/10.1007/978-3-030-49210-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49210-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49209-0

  • Online ISBN: 978-3-030-49210-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation