Skip to main content
SpringerLink
Log in

Cost-Sensitive Extensions for Global Model Trees: Application in Loan Charge-Off Forecasting

  • Conference paper
Advances in Systems Science

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 240))

  • 2228 Accesses

Abstract

Most of regression learning methods aim to reduce various metrics of prediction errors. However, in many real-life applications it is prediction cost, which should be minimized as the under-prediction and over-prediction errors have different consequences. In this paper, we show how to extend the evolutionary algorithm (EA) for global induction of model trees to achieve a cost-sensitive learner. We propose a new fitness function which allows minimization of the average misprediction cost and two specialized memetic operators that search for cost-sensitive regression models in the tree leaves. Experimental validation was performed with bank loan charge-off forecasting data which has asymmetric costs. Results show that Global Model Trees with the proposed extensions are able to effectively induce cost-sensitive model trees with average misprediction cost significantly lower than in popular post-hoc tuning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barros, R.C., Ruiz, D.D., Basgalupp, M.: Evolutionary model trees for handling continuous classes in machine learning. Information Sciences 181, 954–971 (2011)

    Article  Google Scholar 

  2. Barros, R.C., Basgalupp, M.P., Carvalho, A.C., Freitas, A.A.: A Survey of Evolutionary Algorithms for Decision-Tree Induction. IEEE Transactions on Systems Man and Cybernetics, Part C 42(3), 291–312 (2012)

    Article  Google Scholar 

  3. Bansal, G., Sinha, A.P., Zhao, H.: Tuning data mining methods for cost-sensitive regression: a study in loan charge-off forecasting. Journal of Management Information Systems 25(3), 317–338 (2008)

    Article  Google Scholar 

  4. Bradford, J., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.E.: Pruning decision trees with misclassification costs. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 131–136. Springer, Heidelberg (1998)

    Google Scholar 

  5. Cain, M., Janssen, C.: Real estate price prediction under asymmetric loss. Annals of the Institute of Statistical Mathematics 47(3), 401–414 (1995)

    Google Scholar 

  6. Crone, S.F., Lessmann, S., Stahlbock, R.: Utility based data mining for time series analysis: Cost-sensitive learning for neural network predictors. In: Proc. of 1st UDBM, Chicago, IL, pp. 59–68 (2005)

    Google Scholar 

  7. Czajkowski, M., Kretowski, M.: An Evolutionary Algorithm for Global Induction of Regression Trees with Multivariate Linear Models. In: Kryszkiewicz, M., Rybinski, H., Skowron, A., Raś, Z.W. (eds.) ISMIS 2011. LNCS, vol. 6804, pp. 230–239. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  8. Czajkowski, M., Kretowski, M.: Does Memetic Approach Improve Global Induction of Regression and Model Trees? In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) SIDE 2012 and EC 2012. LNCS, vol. 7269, pp. 174–181. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  9. Domingos, P.: MetaCost: A general method for making classifiers cost-sensitive. In: Proc. of KDD 1999, pp. 155–164. ACM Press (1999)

    Google Scholar 

  10. Elkan, C.: The Foundations of Cost-Sensitive Learning. In: Proc. of IJCAI, pp. 973–978 (2001)

    Google Scholar 

  11. Gagne, P., Dayton, C.M.: Best Regression Model Using Information Criteria. Journal of Modern Applied Statistical Methods 1, 479–488 (2002)

    Google Scholar 

  12. Gendreau, M., Potvin, J.Y.: Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 146 (2010)

    Google Scholar 

  13. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Data Mining, Inference, and Prediction, 2nd edn. Springer (2009)

    Google Scholar 

  14. Kahneman, D., Tversky, A.: Prospect Theory: An Analysis of Decisions under Risk. Econometrica 47(2), 263–292 (1979)

    Article  MATH  Google Scholar 

  15. Kotsiantis, S.B.: Decision trees: a recent overview. Artificial Intelligence Review, 1–23 (2011)

    Google Scholar 

  16. Krętowski, M., Grześ, M.: Evolutionary Induction of Cost-Sensitive Decision Trees. In: Esposito, F., Raś, Z.W., Malerba, D., Semeraro, G. (eds.) ISMIS 2006. LNCS (LNAI), vol. 4203, pp. 121–126. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  17. Kretowski, M., Grześ, M.: Evolutionary Induction of Mixed Decision Trees. International Journal of Data Warehousing and Mining 3(4), 68–82 (2007)

    Article  Google Scholar 

  18. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer (1996)

    Google Scholar 

  19. Murthy, S.: Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery 2, 345–389 (1998)

    Article  Google Scholar 

  20. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical Recipes in C. Cambridge University Press (1988)

    Google Scholar 

  21. Rokach, L., Maimon, O.Z.: Top-down induction of decision trees classifiers - A survey. IEEE Transactions on Systems Man and Cybernetics, Part C 35(4), 476–487 (2005)

    Article  Google Scholar 

  22. Rokach, L., Maimon, O.Z.: Data mining with decision trees: theory and application. Machine Perception Arfitical Intelligence 69 (2008)

    Google Scholar 

  23. Schwarz, G.: Estimating the Dimension of a Model. The Annals of Statistics 6, 461–464 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  24. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Parallel Distributed Processing, pp. 318–362. MIT Press, Cambridge (1986)

    Google Scholar 

  25. Shefrin, H., Statman, M.: The Disposition to Sell Winners Too Early and Ride Losers Too Long: Theory and Evidence. Journal of Finance 40, 777–790 (1985)

    Article  Google Scholar 

  26. Ting, K.: An instance-weighting method to induce cost-sensitive trees. IEEE Transactions on Knowledge and Data Engineering 14(3), 659–665 (2002)

    Article  Google Scholar 

  27. Torgo, L., Ribeiro, R.: Utility-based regression. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 597–604. Springer, Heidelberg (2007)

    Google Scholar 

  28. Turney, P.: Types of cost in inductive concept learning. In: Proc. of ICML 2000 Workshop on Cost-Sensitive Learning, Stanford, CA (2000)

    Google Scholar 

  29. Quinlan, J.: Learning with Continuous Classes. In: Proc. of AI 1992, pp. 343–348. World Scientific (1992)

    Google Scholar 

  30. Varian, H.R.: A Bayesian Approach to Real Estate Assessment. In: Fienberg, S.E., Zellner, A. (eds.) Studies in Bayesian Econometrics and Statistics: In honor of L.J. Savage, North-Holland, Amsterdam, pp. 195–208 (1974)

    Google Scholar 

  31. Zhao, H., Sinha, A.P., Bansal, G.: An extended tuning method for cost-sensitive regression and forecasting. Decision Support Systems 51, 372–383 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Bialystok University of Technology, Wiejska 45a, 15-351, Bialystok, Poland

    Marcin Czajkowski & Marek Kretowski

  2. Collegium of Management and Finance, Warsaw School of Economics, Al. Niepodleglosci 162, 02-554, Warsaw, Poland

    Monika Czerwonka

Authors
  1. Marcin Czajkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monika Czerwonka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marek Kretowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcin Czajkowski .

Editor information

Editors and Affiliations

  1. Institute of Informatics, Wroclaw University of Technology, Wrocław, Poland

    Jerzy Swiątek

  2. Institute of Informatics, Wroclaw University of Technology, Wrocław, Poland

    Adam Grzech

  3. Institute of Informatics, Wroclaw University of Technology, Wrocław, Poland

    Paweł Swiątek

  4. Institute of Informatics, Wroclaw University of Technology, Wrocław, Poland

    Jakub M. Tomczak

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Czajkowski, M., Czerwonka, M., Kretowski, M. (2014). Cost-Sensitive Extensions for Global Model Trees: Application in Loan Charge-Off Forecasting. In: Swiątek, J., Grzech, A., Swiątek, P., Tomczak, J. (eds) Advances in Systems Science. Advances in Intelligent Systems and Computing, vol 240. Springer, Cham. https://doi.org/10.1007/978-3-319-01857-7_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-01857-7_30

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-01856-0

  • Online ISBN: 978-3-319-01857-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation