Abstract
The redundant sources problem in multi-source learning always exists in various real-world applications such as multimedia analysis, information retrieval, and medical diagnosis, in which the heterogeneous representations from different sources always have three-way redundancies. More seriously, the redundancies will cost a lot of storage space, cause high computational time, and degrade the performance of learner. This paper is an attempt to jointly reduce redundant sources. Specifically, a novel Heterogeneous Manifold Smoothness Learning (HMSL) model is proposed to linearly map multi-source data to a low-dimensional feature-isomorphic space, in which the information-correlated representations are close along manifold while the semantic-complementary instances are close in Euclidean distance. Furthermore, to eliminate three-way redundancies, we present a new Correlation-based Multi-source Redundancy Reduction (CMRR) method with 2,1-norm equation and generalized elementary transformation constraints to reduce redundant sources in the learned feature-isomorphic space. Comprehensive empirical investigations are presented that confirm the promise of our proposed framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Mach. Learn. 73(3), 243–272 (2008)
Bakry, A., Elgammal, A.: MKPLS: manifold kernel partial least squares for lipreadingand speaker identification. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 684–691 (2013)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)
Bellman, R.: Dynamic programming and lagrange multipliers. Proc. Natl. Acad. Sci. U.S.A. 42(10), 767 (1956)
Chen, D., Zhao, S., Zhang, L., Yang, Y., Zhang, X.: Sample pair selection for attribute reduction with rough set. IEEE Trans. Knowl. Data Eng. 24(11), 2080–2093 (2012)
Freedman, D.: Efficient simplicial reconstructions of manifolds from their samples. IEEE Trans. Pattern Anal. Mach. Intell. 24(10), 1349–1357 (2002)
Geng, B., Tao, D., Xu, C., Yang, L., Hua, X.: Ensemble manifold regularization. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1227–1233 (2012)
Guillaumin, M., Verbeek, J., Schmid, C.: Multimodal semi-supervised learning for image classification. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 902–909 (2010)
Guo, Y., Xiao, M.: Cross language text classification via subspace co-regularized multi-view learning. In: Proceedings of the ACM International Conference on Machine Learning, pp. 915–922 (2012)
He, X., Li, L., Roqueiro, D., Borgwardt, K.: Multi-view spectral clustering on conflicting views. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017, Part II. LNCS (LNAI), vol. 10535, pp. 826–842. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71246-8_50
Huiskes, M.J., Lew, M.S.: The MIR Flickr retrieval evaluation. In: Proceedings of the ACM International Conference on Multimedia Information Retrieval, pp. 39–43 (2008)
Lan, C., Huan, J.: Reducing the unlabeled sample complexity of semi-supervised multi-view learning. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 627–634 (2015)
Li, Z., Tang, J.: Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Trans. Image Process. 24(12), 5343–5355 (2015)
Luo, P., Peng, J., Guan, Z., Fan, J.: Multi-view semantic learning for data representation. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Soares, C., Gama, J., Jorge, A. (eds.) ECML PKDD 2015, Part I. LNCS (LNAI), vol. 9284, pp. 367–382. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23528-8_23
Meyer, C.D.: Matrix Analysis and Applied Linear Algebra. SIAM, Philadelphia (2000)
Nesterov, Y.: Introductory Lectures on Convex Optimization, vol. 87. Springer, New York (2004). https://doi.org/10.1007/978-1-4419-8853-9
Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient and robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 1813–1821 (2010)
Quanz, B., Huan, J., Mishra, M.: Knowledge transfer with low-quality data: a feature extraction issue. IEEE Trans. Knowl. Data Eng. 24(10), 1789–1802 (2012)
Rasiwasia, N., et al.: A new approach to cross-modal multimedia retrieval. In: Proceedings of the ACM International Conference on Multimedia, pp. 251–260 (2010)
Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. ACM Trans. Graph. 27(3), 16 (2008)
Shahrian, E., Rajan, D.: Weighted color and texture sample selection for image matting. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 718–725 (2012)
Su, H., Yin, Z., Kanade, T., Huh, S.: Active sample selection and correction propagation on a gradually-augmented graph. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 1975–1983 (2015)
Sun, L., Ji, S., Ye, J.: Canonical correlation analysis for multilabel classification: a least-squares formulation, extensions, and analysis. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 194–200 (2011)
Wang, D., Nie, F., Huang, H.: Feature selection via global redundancy minimization. IEEE Trans. Knowl. Data Eng. 27(10), 2743–2755 (2015)
Wang, X., Dong, L., Yan, J.: Maximum ambiguity-based sample selection in fuzzy decision tree induction. IEEE Trans. Knowl. Data Eng. 24(8), 1491–1505 (2012)
Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009)
Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142(1), 397–434 (2012). https://doi.org/10.1007/s10107-012-0584-1
Zhang, L., et al.: Collaborative multi-view denoising. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2045–2054 (2016)
Zhuang, Y., Yang, Y., Wu, F., Pan, Y.: Manifold learning based cross-media retrieval: a solution to media object complementary nature. J. VLSI Signal Process. 46(2–3), 153–164 (2007)
Acknowledgment
This work was supported in part by National Natural Science Foundation of China (No. 61601458).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, L., Wang, S., Jin, X., Jia, S. (2020). Joint Multi-source Reduction. In: Brefeld, U., Fromont, E., Hotho, A., Knobbe, A., Maathuis, M., Robardet, C. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2019. Lecture Notes in Computer Science(), vol 11906. Springer, Cham. https://doi.org/10.1007/978-3-030-46150-8_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-46150-8_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-46149-2
Online ISBN: 978-3-030-46150-8
eBook Packages: Computer ScienceComputer Science (R0)
