Abstract
In this paper, we declare the equivalence between the principal component analysis and the nearest q-flat in the least square sense by showing that, for given m data points, the linear manifold with nearest distance is identical to the linear manifold with largest variance. Furthermore, from this observation, we give a new simpler proof for the approach to find the nearest q-flat.
This is a preview of subscription content, log in via an institution to check access.
References
Bradley, P., Mangasarian, O.: k-plane clustering. J. Glob. Optim. 16(1), 23–32 (2000)
Tseng, P.: Nearest q-flat to m points. J. Optim. Theory Appl. 105(1), 249–252 (2000)
Zhang, T., Szlam, A., Wang, Y., Lerman, G.: Randomized hybrid linear modeling by local best-fit flats. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1927–1934, 2010
Chen, G., Lerman, G.: Spectral curvature clustering (SCC). Int. J. Comput. Vis. 81(3), 317–330 (2009)
Chen, G., Lerman, G.: Foundations of a multi-way spectral clustering framework for hybrid linear modeling. Found. Comput. Math. 9(5), 517–558 (2009)
Wang, Y., Jiang, Y., Wu, Y., Zhou, Z.H.: Spectral clustering on multiple manifolds. IEEE Trans. Neural Netw. 22(7), 1149–1161 (2011)
Shao, Y.H., Bai, L., Wang, Z., Hua, X.Y., Deng, N.Y.: Proximal plane clustering via eigenvalues. Proc. Comput. Sci. 17, 41–47 (2013)
Amaldi, E., Dhyani, K., Liberti, L.: A two-phase heuristic for the bottleneck k-hyperplane clustering problem. Comput. Optim. Appl. 56(3), 619–633 (2013)
Szlam, A., Sapiro, G.: Discriminative k-metrics. In: 2009 ACM Conference on Proceedings of the International Conference on Machine Learning (ICML), pp. 1009–1016, 2009
Lerman, G., Zhang, T.: Robust recovery of multiple subspaces by geometric lp minimization. Ann. Stat. 39(5), 2686–2715 (2011)
Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3501–3508, 2010
Thiagarajan, J.J., Ramamurthy, K.N., Spanias, A.: Multilevel dictionary learning for sparse representation of images. In: 2011 IEEE Digital Signal Processing Workshop and IEEE Signal Processing Education Workshop (DSP/SPE), pp. 271–276, 2011
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. 2(4), 433–459 (2010)
Jolliffe, I.: Principal Component Analysis. Wiley, New York (2005)
Ringnér, M.: What is principal component analysis? Nat. Biotechnol. 26(3), 303–304 (2008)
Demšar, U., Harris, P., Brunsdon, C., Fotheringham, A.S., McLoone, S.: Principal component analysis on spatial data: an overview. Ann. Assoc. Am. Geogr. 103(1), 106–128 (2013)
Acknowledgments
We thank anonymous referees for their detailed comments to improve the paper. This work is supported by the National Natural Science Foundation of China (Nos.11201426 and 11371365), the Zhejiang Provincial Natural Science Foundation of China (Nos.LQ12A01020, LQ13F030010, and LQ14G010004) and the Ministry of Education, Humanities and Social Sciences Research Project of China (No.13YJC910011).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shao, YH., Deng, NY. The Equivalence Between Principal Component Analysis and Nearest Flat in the Least Square Sense. J Optim Theory Appl 166, 278–284 (2015). https://doi.org/10.1007/s10957-014-0647-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-014-0647-y
Keywords
- Linear manifold
- Unsupervised learning
- Nearest q-flat
- Principal component analysis
- Eigenvalue decomposition