Abstract
Regression and forecasting can be viewed as learning the functions with the appropriate input and output variables from the data. To capture the complex relationship among the variables, different techniques like, kernelized dictionary learning are being explored in the existing literature. In this paper, the transform learning based function approximation is presented which has computational and performance advantages over dictionary based techniques. Apart from providing the formulation and derivation of the necessary update steps, the performance results obtained with both synthetic and real data are presented in the paper. The initial results obtained with both the basic and kernelized versions demonstrate the usefulness of the proposed technique for regression and forecasting tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tsanas, A., Xifara, A.: Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)
Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). https://doi.org/10.1109/TSP.2006.881199
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013). https://doi.org/10.1109/TPAMI.2013.50
Cao, Y.: Multivariant Kernel Regression and Smoothing, March 2008. https://in.mathworks.com/matlabcentral/fileexchange/19279
Chen, G., Needell, D.: Compressed sensing and dictionary learning. Finite Frame Theory: Complet. Introd. Overcompleteness 73, 201 (2016)
Ganti, R., Willett, R.M.: Sparse Linear regression with missing data. ArXiv e-prints (2015)
Guo, J., Guo, Y., Kong, X., Zhang, M., He, R.: Discriminative analysis dictionary learning. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI 2016, pp. 1617–1623. AAAI Press (2016), http://dl.acm.org/citation.cfm?id=3016100.3016125
Hsieh, W.W.: Nonlinear canonical correlation analysis by neural networks. Neural Netw. 13(10), 1095–1105 (2000). https://doi.org/10.1016/S0893-6080(00)00067-8
Hsieh, W.W.: Machine Learning Methods in the Environmental Sciences: Neural Networks and Kernels, 1st edn. Cambridge University Press, New York (2009)
Jiang, Z., Lin, Z., Davis, L.S.: Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2651–2664 (2013). https://doi.org/10.1109/TPAMI.2013.88
Kumar, K., Majumdar, A., Chandra, M.G., Kumar, A.A.: Regressing kernel dictionary learning. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), April 2018
Maggu, J., Majumdar, A.: Kernel transform learning. Pattern Recognit. Lett. 98, 117–122 (2017). https://doi.org/10.1016/j.patrec.2017.09.002
Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., Bach, F.R.: Supervised dictionary learning. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 21, pp. 1033–1040. Curran Associates, Inc. (2009). http://papers.nips.cc/paper/3448-supervised-dictionary-learning.pdf
Nguyen, H.V., Patel, V.M., Nasrabadi, N.M., Chellappa, R.: Kernel dictionary learning. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2021–2024, March 2012. https://doi.org/10.1109/ICASSP.2012.6288305
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by v1? Vis. Res. 37(23), 3311–3325 (1997). https://doi.org/10.1016/S0042-6989(97)00169-7, http://www.sciencedirect.com/science/article/pii/S0042698997001697
Peng, C., Cheng, H., Ko, M.: An efficient two-stage sparse representation method. Int. J. Pattern Recognit. Artif Intell. 30(01), 1651001 (2016)
Ravishankar, S., Bresler, Y.: Learning doubly sparse transforms for images. IEEE Trans. Image Process. 22(12), 4598–4612 (2013). https://doi.org/10.1109/TIP.2013.2274384
Ravishankar, S., Bresler, Y.: Learning overcomplete sparsifying transforms for signal processing. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3088–3092, May 2013. https://doi.org/10.1109/ICASSP.2013.6638226
Ravishankar, S., Bresler, Y.: Learning sparsifying transforms. IEEE Trans. Signal Process. 61(5), 1072–1086 (2013). https://doi.org/10.1109/TSP.2012.2226449
Ravishankar, S., Bresler, Y.: Efficient blind compressed sensing using sparsifying transforms with convergence guarantees and application to magnetic resonance imaging. SIAM J. Imaging Sci. 8(4), 2519–2557 (2015). https://doi.org/10.1137/141002293
Ravishankar, S., Bresler, Y.: Online sparsifying transform learning part II: convergence analysis. IEEE J. Sel. Top. Signal Process. 9(4), 637–646 (2015). https://doi.org/10.1109/JSTSP.2015.2407860
Ravishankar, S., Wen, B., Bresler, Y.: Online sparsifying transform learning part I: algorithms. IEEE J. Sel. Top. Signal Process. 9(4), 625–636 (2015). https://doi.org/10.1109/JSTSP.2015.2417131
Tang, W., Otero, I.R., Krim, H., Dai, L.: Analysis dictionary learning for scene classification. In: 2016 IEEE Statistical Signal Processing Workshop (SSP), pp. 1–5, June 2016. https://doi.org/10.1109/SSP.2016.7551849
Tosic, I., Frossard, P.: Dictionary learning: what is the right representation for my signal? IEEE Signal Process. Mag. 28(2), 27–38 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kumar, K., Majumdar, A., Girish Chandra, M., Anil Kumar, A. (2020). Transform Learning Based Function Approximation for Regression and Forecasting. In: Lemaire, V., Malinowski, S., Bagnall, A., Bondu, A., Guyet, T., Tavenard, R. (eds) Advanced Analytics and Learning on Temporal Data. AALTD 2019. Lecture Notes in Computer Science(), vol 11986. Springer, Cham. https://doi.org/10.1007/978-3-030-39098-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-39098-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39097-6
Online ISBN: 978-3-030-39098-3
eBook Packages: Computer ScienceComputer Science (R0)
