Abstract
A new iterative procedure for solving regression problems with the so-called LASSO penalty [1] is proposed by using generative Bayesian modeling and inference. The algorithm produces the anticipated parsimonious or sparse regression models that generalize well on unseen data. The proposed algorithm is quite robust and there is no need to specify any model hyperparameters. A comparison with state-of-the-art methods for constructing sparse regression models such as the relevance vector machine (RVM) and the local regularization assisted orthogonal least squares regression (LROLS) is given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. Royal. Statist. Soc. B 58, 267–288 (1996)
Meinshausen, N.: Relaxed LASSO. Computational Statistics & Data Analysis 52(1), 374–393 (2007)
Figueiredo, M., Nowak, R., Wright, S.: Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing 1(4), 586–597 (2007)
Hesterberg, T., Choi, N., Meier, L., Fraley, C.: Least angle and L1 regression: A Review. Statistics Surveys 2, 61–93 (2008)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Annals of Statistics 32, 407–451 (2004)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Schölkopf, B., Smola, A.: Learning with Kernels. The MIT Press, Cambridge (2002)
Chen, S.: Local regularization assisted orthogonal least squares regression. NeuroComputing 69, 559–585 (2006)
Drezet, P., Harrison, R.: Support vector machines for system identification. In: Proceeding of UKACC Int. Conf. Control 1998, Swansea, U.K., pp. 688–692 (1998)
Tipping, M.: Sparse Bayesian learning and the relevance vector machine. J. Machine Learning Research 1, 211–244 (2001)
Chen, S., Billings, S., Luo, W.: Orthogonal least squares methods and their application to non-linear system identification. International Journal of Control 50(5), 1873–1896 (1989)
Kruif, B., Vries, T.: Support-Vector-based least squares for learning non-linear dynamics. In: Proceedings of 41st IEEE Conference on Decision and Control, Las Vegas, USA, pp. 10–13 (2002)
Gestel, T., Espinoza, M., Suykens, J., Brasseur, C., deMoor, B.: Bayesian input selection for nonlinear regression with LS-SVMS. In: Proceedings of 13th IFAC Symposium on System Identification, Totterdam, The Netherlands, pp. 27–29 (2003)
Valyon, J., Horváth, G.: A generalized LS-SVM. In: Principe, J., Gile, L., Morgan, N., Wilson, E. (eds.) Proceedings of 13th IFAC Symposium on System Identification, Rotterdam, The Netherlands (2003)
Suykens, J., van Gestel, T., DeBrabanter, J., DeMoor, B.: Least Square Support Vector Machines. World Scientific, Singapore (2002)
Pontil, M., Mukherjee, S., Girosi, F.: On the noise model of support vector machine regression. A.I. Memo 1651, AI Laboratory, MIT (1998)
Gao, J., Gunn, S., Kandola, J.: Adapting kernels by variational approach in SVM. In: McKay, B., Slaney, J.K. (eds.) Canadian AI 2002. LNCS (LNAI), vol. 2557, pp. 395–406. Springer, Heidelberg (2002)
Gao, J., Xu, R.: Mixture of the robust L1 distributions and its applications. In: Orgun, M.A., Thornton, J. (eds.) AI 2007. LNCS (LNAI), vol. 4830, pp. 26–35. Springer, Heidelberg (2007)
Gao, J.: Robust L1 principal component analysis and its Bayesian variational inference. Neural Computation 20, 555–572 (2008)
Billings, S., Chen, S., Backhouse, R.: The identification of linear and nonlinear models of a turbocharged automotive diesel engine. Mech. Syst. Signal Processing 3(2), 123–142 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, J., Antolovich, M., Kwan, P.W. (2008). L1 LASSO Modeling and Its Bayesian Inference. In: Wobcke, W., Zhang, M. (eds) AI 2008: Advances in Artificial Intelligence. AI 2008. Lecture Notes in Computer Science(), vol 5360. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89378-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-540-89378-3_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-89377-6
Online ISBN: 978-3-540-89378-3
eBook Packages: Computer ScienceComputer Science (R0)