Skip to main content
SpringerLink
Log in

Global Induction of Oblique Model Trees: An Evolutionary Approach

  • Conference paper
Artificial Intelligence and Soft Computing (ICAISC 2013)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7895))

Included in the following conference series:

Abstract

In this paper we propose a new evolutionary algorithm for global induction of oblique model trees that associates leaves with multiple linear regression models. In contrast to the typical top-down approaches it globally searches for the best tree structure, splitting hyper-planes in internal nodes and models in the leaves. The general structure of proposed solution follows a typical framework of evolutionary algorithms with an unstructured population and a generational selection. We propose specialized genetic operators to mutate and cross-over individuals (trees). The fitness function is based on the Bayesian Information Criterion. In preliminary experimental evaluation we show the impact of the tree representation on solving different prediction problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barros, R.C., Ruiz, D.D., Basgalupp, M.: Evolutionary model trees for handling continuous classes in machine learning. Information Sciences 181, 954–971 (2011)

    Article  Google Scholar 

  2. Barros, R.C., Basgalupp, M.P., Carvalho, A.C., Freitas, A.A.: A Survey of Evolutionary Algorithms for Decision-Tree Induction. IEEE Transactions on Systems Man and Cybernetics, Part C 42(3), 291–312 (2012)

    Article  Google Scholar 

  3. Blake, C., Keogh, E., Merz, C.: UCI Repository of Machine Learning Databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

  4. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth Int. Group (1984)

    Google Scholar 

  5. Czajkowski, M., Kretowski, M.: Globally Induced Model Trees: An Evolutionary Approach. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 324–333. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  6. Czajkowski, M., Kretowski, M.: An Evolutionary Algorithm for Global Induction of Regression Trees with Multivariate Linear Models. In: Kryszkiewicz, M., Rybinski, H., Skowron, A., Raś, Z.W. (eds.) ISMIS 2011. LNCS, vol. 6804, pp. 230–239. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  7. Czajkowski, M., Kretowski, M.: Does Memetic Approach Improve Global Induction of Regression and Model Trees? In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) EC 2012 and SIDE 2012. LNCS, vol. 7269, pp. 174–181. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  8. Dobra, A., Gehrke, J.: SECRET: A Scalable Linear Regression Tree Algorithm. In: Proc. of KDD (2002)

    Google Scholar 

  9. Fan, G., Gray, J.B.: Regression tree analysis using target. Journal of Computational and Graphical Statistics 14(1), 206–218 (2005)

    Article  MathSciNet  Google Scholar 

  10. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.): Advances in Knowledge Discovery and Data Mining. AAAI Press (1996)

    Google Scholar 

  11. Gagne, P., Dayton, C.M.: Best Regression Model Using Information Criteria. Journal of Modern Applied Statistical Methods 1, 479–488 (2002)

    Google Scholar 

  12. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Data Mining, Inference, and Prediction, 2nd edn. Springer (2009)

    Google Scholar 

  13. Karalic, A.: Linear Regression in Regression Tree Leaves. International School for Synthesis of Expert Knowledge, Bled, Slovenia (1992)

    Google Scholar 

  14. Kotsiantis, S.B.: Decision trees: a recent overview. Artificial Intelligence Review, 1–23 (2011)

    Google Scholar 

  15. Kretowski, M., Grzes, M.: Global induction of oblique decision trees: An evolutionary approach, Intelligent Information Processing and Web Mining. In: Proc. of the IIS. Advances in Soft Computing, pp. 309–318 (2005)

    Google Scholar 

  16. Kretowski, M., Czajkowski, M.: An Evolutionary Algorithm for Global Induction of Regression Trees. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 157–164. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  17. Li, K.C., Lue, H.H., Chen, C.H.: Interactive Tree-Structured Regression via Principal Hessian Directions. Journal of the American Statistical Association 95, 547–560 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  18. Llorà, X., Wilson, S.W.: Mixed decision trees: Minimizing knowledge representation bias in LCS. In: Deb, K., Tari, Z. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 797–809. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  19. Malerba, D., Esposito, F., Ceci, M., Appice, A.: Top-down Induction of Model Trees with Regression and Splitting Nodes. IEEE Trans. on PAMI 26(5), 612–625 (2004)

    Article  Google Scholar 

  20. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer (1996)

    Google Scholar 

  21. Naumov, G.E.: NP-completeness of problems of construction of optimal decision trees. Soviet Physics Doklady 36(4), 270–271 (1991)

    MathSciNet  MATH  Google Scholar 

  22. Potts, D., Sammut, C.: Incremental Learning of Linear Model Trees. Machine Learning 62, 5–48 (2005)

    Article  Google Scholar 

  23. Potgieter, G., Engelbrecht, A.: Evolving model trees for mining data sets with continuous-valued classes. Expert Systems with Applications 35, 1513–1532 (2008)

    Article  Google Scholar 

  24. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical Recipes in C. Cambridge University Press (1988)

    Google Scholar 

  25. Rokach, L., Maimon, O.: Top-down induction of decision trees classifiers - A survey. IEEE Transactions on Systems, Man, and Cybernetics - Part C 35(4), 476–487 (2005)

    Article  Google Scholar 

  26. Rokach, L., Maimon, O.Z.: Data mining with decision trees: theory and application. Machine Perception Arfitical Intelligence 69 (2008)

    Google Scholar 

  27. Schwarz, G.: Estimating the Dimension of a Model. The Annals of Statistics 6, 461–464 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  28. Torgo, L.: Functional Models for Regression Tree Leaves. In: Proc. of ICML, pp. 385–393. Morgan Kaufmann (1997)

    Google Scholar 

  29. Quinlan, J.: Learning with Continuous Classes. In: Proc. of AI 1992, pp. 343–348. World Scientific (1992)

    Google Scholar 

  30. Vogel, D., Asparouhov, O., Scheffer, T.: Scalable look-ahead linear regression trees. In: Proc. of 13th ACM SIGKDD, pp. 757–764. ACM Press, New York (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Bialystok University of Technology, Wiejska 45a, 15-351, Białystok, Poland

    Marcin Czajkowski & Marek Kretowski

Authors
  1. Marcin Czajkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marek Kretowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Armii Krajowej 36, 42-200, Częstochowa, Poland

    Leszek Rutkowski , Marcin Korytkowski  & Rafał Scherer ,  & 

  2. AGH University of Science and Technology, Michiewicza 30, 30-059, Kraków, Poland

    Ryszard Tadeusiewicz

  3. Department of Electrical Engineering and Computer Sciences, University of California, 94720-1776, Berkeley, CA, USA

    Lotfi A. Zadeh

  4. Electrical and Computer Engineering, University of Louisville, 405 Lutz Hall, 40292, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Czajkowski, M., Kretowski, M. (2013). Global Induction of Oblique Model Trees: An Evolutionary Approach. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2013. Lecture Notes in Computer Science(), vol 7895. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38610-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38610-7_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38609-1

  • Online ISBN: 978-3-642-38610-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation