Abstract
The core tasks of graph based semi-supervised learning (GSSL) are constructing a proper graph and selecting suitable supervisory information. The ideal graph is able to outline the intrinsic data structure, and the ideal supervisory information could represent the whole data. In this paper, we propose a new graph learning method, called constrained least squares regression (CLSR), which integrates the supervisory information into graph learning process. To learn a more adaptive graph, regression coefficients and neighbor relations are combined in CLSR to capture the global and local data structures respectively. Moreover, as byproduct of CLSR, a new strategy is presented to select the high-quality data points as labeled samples, which is practical in real applications. Experimental results on different real world datasets demonstrate the effectiveness of CLSR and the sample selection strategy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15(6), 1373–1396 (2003)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. The Journal of Machine Learning Research 7, 2399–2434 (2006)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning 3(1), 1–122 (2011)
Chapelle, O., Weston, J., Schölkopf, B.: Cluster kernels for semi-supervised learning. In: Advances in Neural Information Processing Systems, pp. 585–592 (2002)
Chen, X., Cai, D.: Large scale spectral clustering with landmark-based representation. In: The 25th Conference on Artificial Intelligence, AAAI 2011 (2011)
Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: Proceedings of the 22th Conference on Computer Vision and Pattern Recognition, pp. 2790–2797. IEEE (2009)
Jebara, T., Wang, J., Chang, S.F.: Graph construction and b-matching for semi-supervised learning. In: Proceedings of the 26th International Conference on Machine Learning, pp. 441–448. ACM (2009)
Li, Z., Liu, J., Tang, X.: Pairwise constraint propagation by semidefinite programming for semi-supervised classification. In: Proceedings of the 25th International Conference on Machine Learning, pp. 576–583 (2008)
Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical report UILU-ENG-09-2215 (2010)
Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 171–184 (2013)
Liu, W., He, J., Chang, S.F.: Large graph construction for scalable semi-supervised learning. In: Proceedings of the 27th International Conference on Machine Learning, pp. 679–686 (2010)
Liu, W., Wang, J., Chang, S.F.: Robust and scalable graph-based semisupervised learning. Proceedings of the IEEE 100(9), 2624–2638 (2012)
Lu, C.-Y., Min, H., Zhao, Z.-Q., Zhu, L., Huang, D.-S., Yan, S.: Robust and efficient subspace segmentation via least squares regression. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 347–360. Springer, Heidelberg (2012)
Peng, X., Zhang, L., Yi, Z.: Scalable sparse subspace clustering. In: IEEE Proceedings of the 26th Conference on Computer Vision and Pattern Recognition (2013)
Shang, F., Jiao, L., Liu, Y., Tong, H.: Semi-supervised learning with nuclear norm regularization. Pattern Recognition 46(8), 2323–2336 (2013)
Tenenbaum, J.B., De Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Yan, S., Wang, H.: Semi-supervised learning by sparse representation. In: SDM, pp. 792–801 (2009)
Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: Which helps face recognition? In: Proceedings of the 12th International Conference on Computer Vision, pp. 471–478. IEEE (2011)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. Advances in Neural Information Processing Systems 16(16), 321–328 (2004)
Zhu, X.: Semi-supervised learning literature survey. Technical report, Department of Computer Science, University of Wisconsin-Madison (2006)
Zhu, X., Ghahramani, Z., Lafferty, J., et al.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine Learning, vol. 3, pp. 912–919 (2003)
Zhuang, L., Gao, H., Lin, Z., Ma, Y., Zhang, X., Yu, N.: Non-negative low rank and sparse graph for semi-supervised learning. In: Proceedings of the 25th Conference on Computer Vision and Pattern Recognition, pp. 2328–2335. IEEE (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Liu, B., Jing, L., Yu, J., Li, J. (2014). Constrained Least Squares Regression for Semi-Supervised Learning. In: Tseng, V.S., Ho, T.B., Zhou, ZH., Chen, A.L.P., Kao, HY. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2014. Lecture Notes in Computer Science(), vol 8444. Springer, Cham. https://doi.org/10.1007/978-3-319-06605-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-06605-9_10
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06604-2
Online ISBN: 978-3-319-06605-9
eBook Packages: Computer ScienceComputer Science (R0)