Abstract
An autoencoder is a neural network to realize an identity mapping with hidden layers of a relatively small number of nodes. However, the role of the hidden layers is not clear because they are automatically determined through the learning process. We propose to realize autoencoders by a set of linear combinations of kernels instead of neural networks. In this framework, the roles of the encoder and/or decoder, are explicitly determined by a user. We show that it is possible to replace almost every type of autoencoders realized by neural networks with this approach. We compare the pros and cons of this kernel approach and the neural network approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscipl. Rev. Comput. Stat. 2(4), 433–459 (2010)
Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)
Bank, D., Koenigstein, N., Giryes, R.: Autoencoders. arXiv preprint arXiv:2003.05991 (2020)
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)
Deng, L.: The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process. Mag. 29(6), 141–142 (2012)
Gholami, B., Hajisami, A.: Kernel auto-encoder for semi-supervised hashing. In: 2016 IEEE Winter Conference on Applications ofCComputer Vision (WACV), pp. 1–8. IEEE (2016)
Goodfellow, I.J., et al.: Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Knaf, H.: Kernel fisher discriminant functions - a concise and rigorous introduction. Tech. Rep. 117, Fraunhofer (ITWM) (2007)
Kudo, M., et alEfficient leave-one-out evaluation of kernelized implicit mappings. Accepted in S+SSPR (2022)
Laforgue, P., Clémençon, S., d’Alché Buc, F.: Autoencoding any data through kernel autoencoders. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1061–1069. PMLR (2019)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579-=-2605 (2008)
Mika, S., et al.: Kernel PCA and de-noising in feature spaces. In: NIPS, vol. 11, pp. 536–542 (1998)
Noh, J., et al.: Machine-enabled inverse design of inorganic solid materials: promises and challenges. Chem. Sci. 11(19), 4871–4881 (2020)
Tai, M., et al.: Kernelized supervised laplacian eigenmap for visualization and classification of multi-label data. Pattern Recogn. 123, 108399 (2022)
Vincent, P., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ICML2008, Association for Computing Machinery, New York, NY, USA (2008)
Wang, W., et al.: Generalized autoencoder: a neural network framework for dimensionality reduction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2014)
Acknowledgment
This work was partially supported by JSPS KAKENHI (Grant Number 19H04128).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Morishita, S., Kudo, M., Kimura, K., Sun, L. (2022). Realization of Autoencoders by Kernel Methods. In: Krzyzak, A., Suen, C.Y., Torsello, A., Nobile, N. (eds) Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2022. Lecture Notes in Computer Science, vol 13813. Springer, Cham. https://doi.org/10.1007/978-3-031-23028-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-23028-8_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23027-1
Online ISBN: 978-3-031-23028-8
eBook Packages: Computer ScienceComputer Science (R0)