Abstract
Multi-task learning (MTL) aims to solve multiple related learning tasks simultaneously so that the useful information in one specific task can be utilized by other tasks in order to improve the learning performance of all tasks. Many representative MTL methods have been proposed to characterize the relationship between different learning tasks. However, the existing methods have not explicitly quantified the distance or similarity of different tasks, which is actually of great importance in modeling the task relation for MTL. In this paper, we propose a novel method called Metric-guided MTL (M\(^2\)TL), which explicitly measures the task distance using a metric learning strategy. Specifically, we measure the distance between different tasks using their projection parameters and learn a distance metric accordingly, so that the similar tasks are close to each other while the uncorrelated tasks are faraway from each other, in terms of the learned distance metric. With a metric-guided regularizer incorporated in the proposed objective function, we open a new way to explore the related information among tasks. The proposed method can be efficiently solved via an alternative method. Experiments on both synthetic and real-world benchmark datasets demonstrate the superiority of the proposed method over existing MTL methods in terms of prediction accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Proceedings 19th NIPS, pp. 41–48 (2007)
Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Mach. Learn. 73(3), 243–272 (2008)
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
Chu, X., Ouyang, W., Yang, W., Wang, X.: Multi-task recurrent neural network for immediacy prediction. In: Proceedings 15th ICCV, pp. 3352–3360 (2015)
Deng, D., Shahabi, C., Demiryurek, U., Zhu, L.: Situation aware multi-task learning for traffic prediction. In: Proceedings 17th ICDM, pp. 81–90 (2017)
Goncalves, A., Banerjee, A., Zuben, F.V.: Spatial projection of multiple climate variables using hierarchical multitask learning. In: Proceedings 31th AAAI, pp. 4509–4515 (2017)
Gong, P., Ye, J., Zhang, C.: Robust multi-task feature learning. In: Proceedings 18th SIGKDD, pp. 895–903 (2012)
Jacob, L., Vert, J.P., Bach, F.R.: Clustered multi-task learning: a convex formulation. In: Proceedings 21th NIPS, pp. 745–752 (2009)
Jalali, A., Sanghavi, S., Ruan, C., Ravikumar, P.K.: A dirty model for multi-task learning. In: Proceedings 22th NIPS, pp. 964–972 (2010)
Lahoud, J., Ghanem, B., Pollefeys, M., Oswald, M.R.: 3d instance segmentation via multi-task metric learning (2019). arXiv preprint arXiv:1906.08650
Liu, J., Ji, S., Ye, J.: Multi-task feature learning via efficient l 2, 1-norm minimization. In: Proceedings 25th UAI, pp. 339–348 (2009)
Ni, J., Liu, J., Zhang, C., Ye, D., Ma, Z.: Fine-grained patient similarity measuring using deep metric learning. In: Proceedings 26th CIKM, pp. 1189–1198. ACM (2017)
Obozinski, G., Taskar, B., Jordan, M.: Multi-task feature selection. University of California, Berkeley, Technical report (2006)
Pei, H., B.Yang, Liu, J., Dong, L.: Group sparse Bayesian learning for active surveillance on epidemic dynamics. In: Proceedings 32th AAAI, pp. 800–807 (2018)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B (Methodological) 58(1), 267–288 (1996)
Xing, E.P., Jordan, M.I., Russell, S.J., Ng, A.Y.: Distance metric learning with application to clustering with side-information. In: Proceedings 15th NIPS, pp. 521–528 (2003)
Yu, H.X., Wu, A., Zheng, W.S.: Cross-view asymmetric metric learning for unsupervised person re-identification. In: Proceedings 16th ICCV, pp. 994–1002 (2017)
Zhang, J., Ghahramani, Z., Yang, Y.: Learning multiple related tasks using latent independent component analysis. In: Proceedings 18th NIPS, pp. 1585–1592 (2006)
Zhang, J., Miao, J., Zhao, K., Tian, Y.: Multi-task feature selection with sparse regularization to extract common and task-specific features. Neurocomputing 340, 76–89 (2019)
Zhang, Y., Yang, Q.: A survey on multi-task learning (2018). arXiv:1707.08114v2
Zhang, Y., Yeung, D.Y.: Multi-task boosting by exploiting task relationships. In: Flach, P.A., De Bie, T., Cristianini, N. (eds.) Proceedings 22th ECML PKDD, pp. 697–710 (2012)
Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Proceedings 13th ECCV, pp. 94–108 (2014)
Zhou, J., Chen, J., Ye, J.: Malsar: multi-task learning via structural regularization. Arizona State University, Technical report (2011)
Acknowledgement
The authors would like to thank the anonymous reviewers for their valuable comments and suggestions. This work was supported by the Grants from the Research Grant Council of Hong Kong SAR under Projects RGC/HKBU12201318 and RGC/HKBU12201619.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ren, J., Liu, Y., Liu, J. (2020). Metric-Guided Multi-task Learning. In: Helic, D., Leitner, G., Stettinger, M., Felfernig, A., Raś, Z.W. (eds) Foundations of Intelligent Systems. ISMIS 2020. Lecture Notes in Computer Science(), vol 12117. Springer, Cham. https://doi.org/10.1007/978-3-030-59491-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-59491-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59490-9
Online ISBN: 978-3-030-59491-6
eBook Packages: Computer ScienceComputer Science (R0)