Abstract
In supervised machine learning, some algorithms are restricted to discrete data and need to discretize continuous attributes. The Khiops discretization method, based on chi-square statistics, optimizes the chi-square criterion in a global manner on the whole discretization domain. In this paper, we propose a major evolution of the Khiops algorithm, that provides guarantees against overfitting and thus significantly improve the robustness of the discretizations. This enhancement is based on a statistical modeling of the Khiops algorithm, derived from the study of the variations of the chi-square value during the discretization process. This modeling, experimentally checked, allows to modify the algorithm and to bring a true control of overfitting. Extensive experiments demonstrate the validity of the approach and show that the Khiops method builds high quality discretizations, both in terms of accuracy and of small interval number.
French patents No 01 07006 and No 02 16733
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases Web URL http://www.ics.uci.edu/~mlearn/MLRepository.html. Irvine, CA: University of California, Department of Information and Computer Science (1998)
Boullé, M.: Khiops: une méthode statistique de discrétisation. Extraction des connaissances et apprentissage, Vol 1-no4. Hermes Science Publications (2001) 107–118
Boullé, M.: Amélioration de la robustesse de la méthode Khiops par contrôle de son comportement statistique. Note technique NT/FTR&D/7864. France Telecom R&D (2002)
Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J.: Classification and Regression Trees. California: Wadsworth International (1984)
Dougherty, J., Kohavi, R and Sahami, M.: Supervised and Unsupervised Discretization of Continuous Features. Proceedings of the Twelf International Conference on Machine Learning, Los Altos, CA: Morgan Kaufmann, (1995) 194–202
Fayyad, U., Irani, K.: On the handling of continuous-valued attributes in decision tree generation. Machine Learning, 8 (1992) 87–102
Kass, G.V.: An exploratory technique for investigating large quantities of categorical data. Applied Statistics, 29(2) (1980) 119–127
Kerber, R.: Chimerge discretization of numeric attributes. Proceedings of the 10th International Conference on Artificial Intelligence (1991) 123–128
Liu, H., Hussain, F., Tan, C.L. and Dash, M. Discretization: An Enabling Technique. Data Mining and Knowledge Discovery 6(4) (2002) 393–423
Perner, P., Trautzsch, S.: Multi-interval Discretization Methods for Decision Tree Learning. SSPR/SPR (1998) 475–482
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)
Zighed, D.A., Rabaseda, S. & Rakotomalala, R.: Fusinter: a method for discretization of continuous attributes for supervised learning. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(33) (1998) 307–326.
Zighed, D.A., Rakotomalala, R.: Graphes d’induction. Hermes Science Publications (2000) 327–359
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Boullé, M. (2003). Khiops: A Discretization Method of Continuous Attributes with Guaranteed Resistance to Noise. In: Perner, P., Rosenfeld, A. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2003. Lecture Notes in Computer Science, vol 2734. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45065-3_5
Download citation
DOI: https://doi.org/10.1007/3-540-45065-3_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40504-7
Online ISBN: 978-3-540-45065-8
eBook Packages: Springer Book Archive