Abstract
Typically, an optimal smoothing parameter in a penalized spline regression is determined by minimizing an information criterion, such as one of the C p , CV and GCV criteria. Since an explicit solution to the minimization problem for an information criterion cannot be obtained, it is necessary to carry out an iterative procedure to search for the optimal smoothing parameter. In order to avoid such extra calculation, a non-iterative optimization method for smoothness in penalized spline regression is proposed using the formulation of generalized ridge regression. By conducting numerical simulations, we verify that our method has better performance than other methods which optimize the number of basis functions and the single smoothing parameter by means of the CV or GCV criteria.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Balakrishnan, N., Ambagaspitiya, R.S.: On skew Laplace distribution. Technical Report, Department of Mathematics & Statistics, McMaster University, Hamilton, Ontario, Canada (1994)
Buckley, M.J., Eagleson, G.K., Silverman, B.W.: The estimation of residual variance in nonparametric regression. Biometrika 75, 189–199 (1988)
Craven, P., Wahba, G.: Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31, 377–403 (1979)
Efron, B.: The estimation of prediction error: covariance penalties and cross-validation. J. Am. Stat. Assoc. 99, 619–632 (2004)
Eilers, P., Marx, B.: Flexible smoothing with B-splines and penalties (with discussion). Stat. Sci. 11, 89–121 (1996)
Gasser, T., Sroka, L., Jennen-Steinmetz, C.: Residual variance and residual pattern in nonlinear regression model. Biometrika 73, 625–633 (1986)
Green, P.J., Silverman, B.W.: Nonparametric Regression and Generalized Linear Models. Chapman & Hall/CRC, Boca Raton (1994)
Hall, P., Kay, J.W., Titterington, D.M.: Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika 77, 521–528 (1990)
Härdle, W.: Applied Nonparametric Regression. Cambridge University Press, Cambridge (1990)
Hastie, T.: Pseudosplines. J. R. Stat. Soc. B 58, 379–396 (1996)
Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models. Chapman & Hall/CRC, Boca Raton (1990)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)
Holst, U., Hössjer, O., Björklund, C., Ragnarson, P., Edner, H.: Locally weighted least squares kernel regression and statistical evaluation of LIDAR measurements. Environmetrics 7, 401–416 (1996)
Hurvich, C.M., Simonoff, J.S., Tsai, C.-L.: Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc. B 60, 271–293 (1998)
Imoto, S., Konishi, S.: Selection of smoothing parameters in B-spline nonparametric regression models using information criteria. Ann. Inst. Stat. Math. 55, 671–687 (2003)
Konishi, S., Kitagawa, G.: Generalised information criteria in model selection. Biometrika 83, 875–890 (1996)
Konishi, S., Kitagawa, G.: Information Criteria and Statistical Modeling. Springer, New York (2008)
Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM J. Optim. 9, 112–147 (1998)
Lawless, J.F.: Mean squared error properties of generalized ridge regression. J. Am. Stat. Assoc. 76, 462–466 (1981)
Mallows, C.L.: Some comments on C p . Technometrics 15, 661–675 (1973)
Mallows, C.L.: More comments on C p . Technometrics 37, 362–372 (1995)
Seifert, B., Gasser, T., Wolf, A.: Nonparametric estimation of residual variance revisited. Biometrika 80, 373–383 (1993)
Siotani, M., Hayakawa, T., Fujikoshi, Y.: Modern Multivariate Statistical Analysis: A Graduate Course and Handbook. American Sciences Press, Columbus (1985)
Stone, M.: Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. B 36, 111–147 (1974)
Ueki, M., Fueda, K.: Optimal tuning parameter estimation in maximum penalized likelihood method. Ann. Inst. Stat. Math. 62, 413–438 (2010)
Walker, S.G., Page, C.J.: Generalized ridge regression and a generalization of the C p statistic. J. Appl. Stat. 28, 911–922 (2001)
Wand, M.P.: On the optimal amount of smoothing in penalised spline regression. Biometrika 86, 936–940 (1999)
Wand, M.P.: A comparison of regression spline smoothing procedures. Stat. Comput. 15, 443–462 (2000)
Wand, M.P.: Smoothing and mixed models. Comput. Stat. 18, 223–249 (2003)
Yanagihara, H., Ohtaki, M.: Knot-placement to avoid over fitting in B-spline scedastic smoothing. Commun. Stat., Simul. Comput. 32, 771–785 (2003)
Yanagihara, H., Ohtaki, M.: On avoidance of the over-fitting in the B-spline non-parametric regression model. Jpn. J. Appl. Stat. 33, 51–69 (2004) (in Japanese)
Yanagihara, H., Yuan, K.-H.: Four improved statistics for contrasting means by correcting skewness and kurtosis. Br. J. Math. Stat. Psychol. 58, 209–237 (2005)
Yanagihara, H., Nagai, I., Satoh, K.: A bias-corrected C p criterion for optimizing ridge parameters in multivariate generalized ridge regression. Jpn. J. Appl. Stat. 38, 151–172 (2009) (in Japanese)
Zhang, H.: Multivariate adaptive splines for analysis of longitudinal data. J. Comput. Graph. Stat. 6, 74–91 (1997)
Zhang, H.: Mixed effects multivariate adaptive splines model for the analysis of longitudinal and growth curve data. Stat. Methods Med. Res. 13, 63–82 (2004)
Author information
Authors and Affiliations
Corresponding author
Additional information
The author thanks Dr. Hironori Fujisawa, The Institute of Statistical Mathematics, and Prof. Hirofumi Wakaki, Hiroshima University, for helpful comments on Stein’s Lemma and the non-uniqueness of K +. The author also wishes to thank the two reviewers and the editor for their valuable comments. This research was supported by the Ministry of Education, Science, Sports, and Culture, Grant-in-Aid for Challenging Exploratory Research, #22650058, 2010-2012.
Rights and permissions
About this article
Cite this article
Yanagihara, H. A non-iterative optimization method for smoothness in penalized spline regression. Stat Comput 22, 527–544 (2012). https://doi.org/10.1007/s11222-011-9245-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-011-9245-0