Abstract
We introduce Quantile Boost (QBoost) algorithms which predict conditional quantiles of the interested response for regression and binary classification. Quantile Boost Regression (QBR) performs gradient descent in functional space to minimize the objective function used by quantile regression (QReg). In the classification scenario, the class label is defined via a hidden variable, and the quantiles of the class label are estimated by fitting the corresponding quantiles of the hidden variable. An equivalent form of the definition of quantile is introduced, whose smoothed version is employed as the objective function, which is maximized by gradient ascent in functional space to get the Quantile Boost Classification (QBC) algorithm. Extensive experiments show that QBoost performs better than the original QReg and other alternatives for regression and classification. Furthermore, QBoost is more robust to noisy predictors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bühlmann, P., Hothorn, T.: Boosting Algorithms: Regularization, Prediction and Model Fitting. Statistical Science 22, 477–505 (2007)
Bühlmann, P., Yu, B.: Boosting with the L2 Loss: Regression and Classification. Journal of the American Statistical Association 98, 324–340 (2003)
Dettling, M., Bühlmann, P.: Boosting for Tumor Classification with Gene Expression Data. Bioinformatics 19(9), 1061–1069 (2003)
Dietterich, T.G., Bakiri, G.: Solving Multiclass Learning Problems via Error-Correcting Output Codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)
Duffy, N., Helmbold, D.: Boosting Methods for Regression. Machine Learning 47(2-3), 153–200 (2002)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least Angle Regression. Annals of Statistics 32(2), 407–499 (2004)
Freund, Y., Schapire, R.: A Decision-Theoretic Generalization of On-line Learning and An Application to Boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)
Friedman, J.: Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics 29, 1189–1232 (2001)
Friedman, J., Hastie, T., Tibshirani, R.: Additive Logistic Regression: a Statistical View of Boosting. Annal of Statistics 28(2), 337–407 (2000)
Hall, P., Titterington, D.M., Xue, J.H.: Median-based Classifiers for High-Dimensional Data (2008), http://www.gla.ac.uk/media/media_71170_en.pdf
Hunter, D.R., Lange, K.: Quantile Regression via an MM Algorithm. Journal of Computational & Graphical Statistics (2000)
Koenker, R.: Quantile Regression. Cambridge University Press, Cambridge (2005)
Kordas, G.: Credit Scoring Using Binary Quantile Regression. In: Statistical Data Analysis Based on the L 1-Norm and Related Methods (2002)
Kordas, G.: Smoothed Binary Regression Quantiles. Journal of Applied Econometrics 21(3), 387–407 (2006)
Langford, J., Oliveira, R., Zadrozny, B.: Predicting Conditional Quantiles via Reduction to Classification. In: In Proc. of Uncertainty in Artificical Intelligence (2006)
Li, Y., Zhu, J.: L 1-Norm Quantile Regression. Journal of Computational & Graphical Statistics 17(1), 163–185 (2008)
Wu, Y., Liu, Y.: Variable Selection in Quantile Regression. Statistica Sinica 19, 801–817 (2009)
Zemel, R., Pitassi, T.: A Gradient-based Boosting Algorithm for Regression Problems. In: Proc. of Advances in Neural Information Processing Systems (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zheng, S. (2010). Boosting Based Conditional Quantile Estimation for Regression and Binary Classification. In: Sidorov, G., Hernández Aguirre, A., Reyes GarcÃa, C.A. (eds) Advances in Soft Computing. MICAI 2010. Lecture Notes in Computer Science(), vol 6438. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16773-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-16773-7_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16772-0
Online ISBN: 978-3-642-16773-7
eBook Packages: Computer ScienceComputer Science (R0)