Abstract
In some regression problems, it may be more reasonable to predict intervals rather than precise values. We are interested in finding intervals which simultaneously for all input instances \(x \in \mathcal{X}\) contain a β proportion of the response values. We name this problem simultaneous interval regression. This is similar to simultaneous tolerance intervals for regression with a high confidence level γ ≈ 1 and several authors have already treated this problem for linear regression. Such intervals could be seen as a form of confidence envelop for the prediction variable given any value of predictor variables in their domain. Tolerance intervals and simultaneous tolerance intervals have not yet been treated for the K-nearest neighbor (KNN) regression method. The goal of this paper is to consider the simultaneous interval regression problem for KNN and this is done without the homoscedasticity assumption. In this scope, we propose a new interval regression method based on KNN which takes advantage of tolerance intervals in order to choose, for each instance, the value of the hyper-parameter K which will be a good trade-off between the precision and the uncertainty due to the limited sample size of the neighborhood around each instance. In the experiment part, our proposed interval construction method is compared with a more conventional interval approximation method on six benchmark regression data sets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Statistical Tolerance Regions: Theory, Applications, and Computation. Wiley (2009)
Boos, D.D.: A new method for constructing approximate confidence intervals from m estimates. J. Amer. Statistical Assoc. 75(369), 142–145 (1980)
Frank, A., Asuncion, A.: UCI machine learning repository (2010)
Hahn, G.J., Meeker, W.Q.: Statistical Intervals: A Guide for Practitioners. John Wiley and Sons (1991)
Howe, W.G.: Two-sided tolerance limits for normal populations, some improvements. J. Amer. Statistical Assoc. 64(326), 610–620 (1969)
Kocherginsky, M., He, X., Mu, Y.: Practical confidence intervals for regression quantiles. J. Comp. and Graph. Stat. 14(1), 41–55 (2005)
Koenker, R.: Confidence intervals for regression quantiles. In: Proc. of 5th Symp. Asymp. Stat. (1994)
Koenker, R., Bassett, G.: Regression quantiles. Econometrica 46(1), 33–50 (1978)
Koenker, R., Hallock, K.: Quantile Regression: An Introduction. J. Economic Perspectives 15(4), 43–56 (2001)
Li, Q., Racine, J.S.: Nonparametric econometrics: theory and practice. Princeton University Press (2007)
Lieberman, G.J., Miller Jr., R.G.: Simultaneous tolerance intervals in regression. Biometrika 50(1/2), 155–168 (1963)
Mee, R.W., Eberhardt, K.R., Reeve, C.P.: Calibration and simultaneous tolerance intervals for regression. Technometrics 33(2), 211–219 (1991)
Miller, R.G.: Simultaneous Statistical Inference. Springer, New York (1991)
Nadaraya, E.A.: On estimating regression. Theory of Probability and its Applications 9, 141–142 (1964)
Silverman, B.W.: Density estimation for statistics and data analysis. Chapman and Hall, London (1986)
Tsanas, A., Little, M.A., McSharry, P.E., Ramig, L.O.: Accurate telemonitoring of parkinson’s disease progression by non-invasive speech tests (2009)
Wallis, W.A.: Tolerance intervals for linear regression. In: Proc. Second Berkeley Symp. on Math. Statist. and Prob., pp. 43–51. Univ. of Calif. Press (1951)
Watson, G.S.: Smooth regression analysis. Sankhyā Ser. 26, 359–372 (1964)
Wilson, A.L.: An approach to simultaneous tolerance intervals in regression. Ann. of Math. Stat. 38(5), 1536–1540 (1967)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ghasemi Hamed, M., Serrurier, M., Durand, N. (2012). Simultaneous Interval Regression for K-Nearest Neighbor. In: Thielscher, M., Zhang, D. (eds) AI 2012: Advances in Artificial Intelligence. AI 2012. Lecture Notes in Computer Science(), vol 7691. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35101-3_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-35101-3_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35100-6
Online ISBN: 978-3-642-35101-3
eBook Packages: Computer ScienceComputer Science (R0)