Abstract
This work introduces a new unsupervised representation learning technique called Deep Convolutional Transform Learning (DCTL). By stacking convolutional transforms, our approach is able to learn a set of independent kernels at different layers. The features extracted in an unsupervised manner can then be used to perform machine learning tasks, such as classification and clustering. The learning technique relies on a well-sounded alternating proximal minimization scheme with established convergence guarantees. Our experimental results show that the proposed DCTL technique outperforms its shallow version CTL, on several benchmark datasets.
This work was supported by the CNRS-CEFIPRA project under grant NextGenBP PRC2017.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137, 91–129 (2011)
Chabiron, O., Malgouyres, F., Tourneret, J.: Toward fast transform learning. Int. J. Comput. Vis. 114, 195–216 (2015)
Chan, M.C., Stott, J.P.: Deep-CEE I: fishing for galaxy clusters with deep neural nets. Mon. Not. R. Astron. Soc. 490(4), 5770–5787 (2019)
Chouzenoux, E., Pesquet, J.C., Repetti, A.: A block coordinate variable metric forward-backward algorithm. J. Glob. Optim. 66(3), 457–485 (2016)
Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer Optimization and Its Applications, vol. 69, pp. 185–212. Springer-Verlag, New York (2010). https://doi.org/10.1007/978-1-4419-9569-8_10
D.J.: The Yale face database (1997). http://cvc.yale.edu/projects/yalefaces/yalefaces.html. 1(2), 4
El Gheche, M., Chierchia, G., Frossard, P.: Multilayer network data clustering. IEEE Trans. Signal Inf. Process. Over Netw. 6(1), 13–23 (2020)
Fagot, D., Wendt, H., Févotte, C., Smaragdis, P.: Majorization-minimization algorithms for convolutive NMF with the beta-divergence. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2019), pp. 8202–8206 (2019)
Hannun, A.Y., et al.: Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 25(1), 65 (2019)
Kempter, R., Gerstner, W., Van Hemmen, J.L.: Hebbian learning and spiking neurons. Phys. Rev. E 59(4), 4498 (1999)
Lee, K., Ho, J., Kriegman, D.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 5, 684–698 (2005)
Maggu, J., Chouzenoux, E., Chierchia, G., Majumdar, A.: Convolutional transform learning. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11303, pp. 162–174. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04182-3_15
Martinez, A.M.: The AR face database. CVC Technical Report24 (1998)
Mason, E., Yonel, B., Yazici, B.: Deep learning for radar. In: 2017 IEEE Radar Conference (RadarConf), pp. 1703–1708. IEEE (2017)
Ravishankar, S., Bresler, Y.: Learning sparsifying transforms. IEEE Trans. Signal Process. 61(5), 1072–1086 (2013)
Ravishankar, S., Bresler, Y.: Online sparsifying transform learning - Part II. IEEE J. Sel. Topics Signal Process. 9(4), 637–646 (2015)
Ravishankar, S., Wen, B., Bresler, Y.: Online sparsifying transform learning - Part I. IEEE J. Sel. Topics Signal Process. 9(4), 625–636 (2015)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)
Tang, W., Chouzenoux, E., Pesquet, J., Krim, H.: Deep transform and metric learning network: wedding deep dictionary learning and neural networks. Technical report (2020). https://arxiv.org/pdf/2002.07898.pdf
Taylor, G., Burmeister, R., Xu, Z., Singh, B., Patel, A., Goldstein, T.: Training neural networks without gradients: a scalable ADMM approach. In: International Conference on Machine Learning, pp. 2722–2731 (2016)
Van Gerven, M., Bohte, S.: Artificial neural networks as models of neural information processing. Front. Comput. Neurosci. 11, 114 (2017)
Ye, J.C., Han, Y., Cha, E.: Deep convolutional framelets: a general deep learning framework for inverse problems. SIAM J. Imaging Sci. 11(2), 991–1048 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Maggu, J., Majumdar, A., Chouzenoux, E., Chierchia, G. (2020). Deep Convolutional Transform Learning. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, Cham. https://doi.org/10.1007/978-3-030-63823-8_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-63823-8_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63822-1
Online ISBN: 978-3-030-63823-8
eBook Packages: Computer ScienceComputer Science (R0)