Abstract
This paper proposes a novel way to learn multi-task kernel machines by combining the structure of classical Support Vector Machine (SVM) optimization problem with multi-task covariance functions developed in Gaussian process (GP) literature. Specifically, we propose a multi-task Support Vector Machine that can be trained on data with multiple target variables simultaneously, while taking into account the correlation structure between different outputs. In the proposed framework, the correlation structure between multiple tasks is captured by covariance functions constructed using a Fourier transform, which allows to represent both auto and cross-correlation structure between the outputs. We present a mathematical model and validate it experimentally on a rescaled version of the Jura dataset, a collection of samples representing the amount of seven chemical elements into several locations. The results demonstrate the utility of our modeling framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ben-David, S., Schuller, R.: Exploiting task relatedness for multiple task learning. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT-Kernel 2003. LNCS (LNAI), vol. 2777, pp. 567–580. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45167-9_41
Bielza, C., Li, G., Larranaga, P.: Multi-dimensional classification with Bayesian networks. Int. J. Approximate Reasoning 52(6), 705–727 (2011)
Borchani, H., Varando, G., Bielza, C., Larrañaga, P.: A survey on multi-output regression. Wiley Interdisci. Rev. Data Min. Knowl. Disc. 5(5), 216–233 (2015)
Cai, F., Cherkassky, V.: Generalized SMO algorithm for SVM-based multitask learning. IEEE Trans. Neural Netw. Learn. Syst. 23(6), 997–1003 (2012)
Cressie, N.: Statistics for spatial data. Terra Nova 4(5), 613–617 (1992)
Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117 (2004)
Melki, G., Cano, A., Kecman, V., Ventura, S.: Multi-target support vector regression via correlation regressor chains. Inf. Sci. 415, 53–69 (2017)
Melkumyan, A., Ramos, F.: Multi-kernel gaussian processes. In: Twenty-Second International Joint Conference on Artificial Intelligence (2011)
Micchelli, C.A., Pontil, M.: Kernels for multi-task learning. In: Advances in neural Information Processing Systems, pp. 921–928 (2005)
Vembu, S., Gärtner, T.: Label ranking algorithms: a survey. In: Fürnkranz, J., Hüllermeier, E. (eds.) Preference Learning, pp. 45–64. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14125-6_3
Wackernagel, H.: Multivariate Geostatistics: An Introduction with Applications. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-662-05294-5
Xu, D., Shi, Y., Tsang, I.W., Ong, Y.S., Gong, C., Shen, X.: Survey on multi-output learning. IEEE Trans. Neural Networks Learn. Syst. 31(7), 2409–2429 (2019)
Xu, S., An, X., Qiao, X., Zhu, L.: Multi-task least-squares support vector machines. Multimedia Tools Appl. 71(2), 699–715 (2013). https://doi.org/10.1007/s11042-013-1526-5
Zhang, J., He, Y., Tang, J.: Multi-view multi-task support vector machine. In: Shi, Y., et al. (eds.) ICCS 2018. LNCS, vol. 10861, pp. 419–428. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93701-4_32
Acknowledgement
The authors would like to thank Maruan Al-Shedivat for his precious advises and suggestions and Professor Eric Xing for his guidance while one of the authors was visiting the Machine Learning Department at Carnegie Mellon University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Marcelli, E., De Leone, R. (2020). Multi-kernel Covariance Terms in Multi-output Support Vector Machines. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2020. Lecture Notes in Computer Science(), vol 12566. Springer, Cham. https://doi.org/10.1007/978-3-030-64580-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-64580-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64579-3
Online ISBN: 978-3-030-64580-9
eBook Packages: Computer ScienceComputer Science (R0)