Abstract
In the high dimensional regression and classification, we often need the feature selection and the dimensionality reduction to cope with the huge computational cost and the over-fitting of the parameters. Canonical Correlation Analysis (CCA) and their hierarchical extension (includes Bayesian method) was proposed for this purpose. However, the real data set often violates the assumption of the linearity of CCA. Thus, we need the non-linear extension of them. To solve this problem, we propose the Bayesian mixture of CCA and give the efficient inference algorithm by Gibbs sampling. We show that the proposed method is the scalable natural extension of CCA and RBF type neural networks for the high dimensional non-linear problems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bishop, C.M.: Bayesian PCA. Advances in Neural Information. In: Processing Systems, vol. 11, pp. 382–388 (1999)
Katahira, K., Matsumoto, N., Sugase-Miyamoto, Y., Okanoya, K., Okada, M.: Doubly Sparse Factor Models for Unifying Feature Transformation and Feature Selection. Journal of Physics: Conference Series (in press)
Watanabe, S.: Equations of states in singular statistical estimation. Neural Networks 23, 1 (2010)
Furguson, T.S.: A Bayesian analysis of some nonparametric problems. Annals of Statistics 1, 209–230 (1973)
Griffiths, T., Ghahramani, Z.: Infinite latent feature models and the Indian buffet process. In: Advances in Neural Information Processing System (NIPS), vol. 18 (2005)
Ghahramani, Z., Hinton, G.E.: The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96-1, Dept. of Comp. Sci., Univ. of Toronto (1996)
Ghahramani, Z., Beal, M.J.: Variational Inference for Bayesian Mixtures of Factor Analyzers. In: Advances in Neural Information Processing System (NIPS), vol. 12 (2000)
Xu, L., Jordan, M.I., Hinton, G.E.: An Alternative Model for Mixtures of Experts. In: Advances in Neural Information Processing System (NIPS), vol. 7 (1995)
Sato, M., Ishii, S.: On-line EM Algorithm for the Normalized Gaussian Network. Neural Computation 12(2), 407–432 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hosino, T. (2010). High Dimensional Non-linear Modeling with Bayesian Mixture of CCA. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds) Neural Information Processing. Theory and Algorithms. ICONIP 2010. Lecture Notes in Computer Science, vol 6443. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17537-4_55
Download citation
DOI: https://doi.org/10.1007/978-3-642-17537-4_55
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17536-7
Online ISBN: 978-3-642-17537-4
eBook Packages: Computer ScienceComputer Science (R0)