Abstract
A flag is a sequence of nested subspaces. Flags are ubiquitous in numerical analysis, arising in finite elements, multigrid, spectral, and pseudospectral methods for numerical pde; they arise in the form of Krylov subspaces in matrix computations, and as multiresolution analysis in wavelets constructions. They are common in statistics too—principal component, canonical correlation, and correspondence analyses may all be viewed as methods for extracting flags from a data set. The main goal of this article is to develop the tools needed for optimizing over a set of flags, which is a smooth manifold called the flag manifold, and it contains the Grassmannian as the simplest special case. We will derive closed-form analytic expressions for various differential geometric objects required for Riemannian optimization algorithms on the flag manifold; introducing various systems of extrinsic coordinates that allow us to parameterize points, metrics, tangent spaces, geodesics, distances, parallel transports, gradients, Hessians in terms of matrices and matrix operations; and thereby permitting us to formulate steepest descent, conjugate gradient, and Newton algorithms on the flag manifold using only standard numerical linear algebra.
Similar content being viewed by others
References
Absil, P.A., Amodei, L., Meyer, G.: Two Newton methods on the manifold of fixed-rank matrices endowed with Riemannian quotient geometries. Comput. Stat. 29(3–4), 569–590 (2014)
Absil, P.A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)
Absil, P.A., Mahony, R., Sepulchre, R.: Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Appl. Math. 80(2), 199–220 (2004)
Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008)
Alekseevsky, D., Arvanitoyeorgos, A.: Riemannian flag manifolds with homogeneous geodesics. Trans. Am. Math. Soc. 359(8), 3769–3789 (2007)
Ammar, G., Martin, C.: The geometry of matrix eigenvalue methods. Acta Appl. Math. 5(3), 239–278 (1986)
Axler, S.: Linear Algebra Done Right. Undergraduate Texts in Mathematics, 3rd edn. Springer, Cham (2015)
Balzano, L., Nowak, R., Recht, B.: Online identification and tracking of subspaces from highly incomplete information. In: 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 704–711. IEEE (2010)
Bellman, R., Fan, K.: On systems of linear inequalities in Hermitian matrix variables. In: Proceedings on Symposium Pure Mathematics, vol. VII, pp. 1–11. American Mathematical Society, Providence, RI (1963)
Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry. Pure and Applied Mathematics, vol. 120, 2nd edn. Academic Press Inc, Orlando, FL (1986)
Borel, A.: La cohomologie mod \(2\) de certains espaces homogènes. Comment. Math. Helv. 27, 165–197 (1953)
Borel, A.: Sur la cohomologie des espaces fibrés principaux et des espaces homogènes de groupes de Lie compacts. Ann. Math. 2(57), 115–207 (1953)
Borel, A.: Linear Algebraic Groups. Graduate Texts in Mathematics, vol. 126, 2nd edn. Springer-Verlag, New York (1991)
Borel, A., Serre, J.P.: Groupes de Lie et puissances réduites de Steenrod. Am. J. Math. 75, 409–448 (1953)
Boyd, S., Kim, S.J., Vandenberghe, L., Hassibi, A.: A tutorial on geometric programming. Optim. Eng. 8(1), 67–127 (2007)
Cheeger, J., Ebin, D.G.: Comparison Theorems in Riemannian Geometry. AMS Chelsea Publishing, Providence, RI (2008)
Chern, S.S.: On the characteristic classes of complex sphere bundles and algebraic varieties. Am. J. Math. 75, 565–597 (1953)
Chikuse, Y.: Statistics on Special Manifolds. Lecture Notes in Statistics, vol. 174. Springer-Verlag, New York (2003)
Curtef, O., Dirr, G., Helmke, U.: Riemannian optimization on tensor products of Grassmann manifolds: applications to generalized Rayleigh-quotients. SIAM J. Matrix Anal. Appl. 33(1), 210–234 (2012)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1999)
Ehresmann, C.: Sur la topologie de certains espaces homogènes. Ann. Math. (2) 35(2), 396–443 (1934)
Gabay, D.: Minimizing a differentiable function over a differential manifold. J. Optim. Theory Appl. 37(2), 177–219 (1982)
Goodman, R., Wallach, N.R.: Symmetry, Representations, and Invariants, Graduate Texts in Mathematics, vol. 255. Springer, Dordrecht (2009)
Grassmann, H.: A New Branch of Mathematics. Open Court Publishing Co., Chicago, IL (1995)
Helgason, S.: Differential Geometry, Lie Groups, and Symmetric Spaces, Graduate Studies in Mathematics, vol. 34. American Mathematical Society, Providence, RI (2001)
Helmke, U., Hüper, K., Trumpf, J.: Newton’s method on Graßmann manifolds. Preprint (2007). arxiv:0709.2205
Howe, R., Lee, S.T.: Spherical harmonics on Grassmannians. Colloq. Math. 118(1), 349–364 (2010)
Jordan, J., Helmke, U.: Controllability of the QR-algorithm on Hessenberg flags. In: Proceeding of the Fifteenth International Symposium on Mathematical Theory of Network and Systems (MTNS 2002) (2002)
Kobayashi, S., Nomizu, K.: Foundations of Differential Geometry. Vols. I, II. Wiley Classics Library. Wiley, New York (1996)
Kowalski, O., Szenthe, J.: Erratum: “On the existence of homogeneous geodesics in homogeneous Riemannian manifolds” [Geom. Dedicata 81 (2000), no. 1-3, 209–214; MR1772203 (2001f:53104)]. Geom. Dedicata 84(1–3), 331–332 (2001)
Lim, L.H., Wong, K.S.W., Ye, K.: Numerical algorithms on the affine Grassmannian. SIAM J. Matrix Anal. Appl. 40(2), 371–393 (2019)
Lim, L.H., Ye, K.: Numerical algorithms on the flag manifold. Preprint (2019)
Lundström, E., Eldén, L.: Adaptive eigenvalue computations using Newton’s method on the Grassmann manifold. SIAM J. Matrix Anal. Appl. 23(3), 819–839 (2001/02)
Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Probability and Mathematical Statistics: A Series of Monographs and Textbooks. Academic Press, London (1979)
Massart, E., Absil, P.A.: Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices. SIAM J. Matrix Anal. Appl. 41(1), 171–198 (2020)
Monk, D.: The geometry of flag manifolds. Proc. Lond. Math. Soc. 3(9), 253–286 (1959)
Nicolaescu, L.I.: Lectures on the Geometry of Manifolds. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ (2007)
Nishimori, Y., Akaho, S., Plumbley, M.D.: Riemannian optimization method on the flag manifold for independent subspace analysis. In: International Conference on Independent Component Analysis and Signal Separation, pp. 295–302. Springer (2006)
Nishimori, Y., Akaho, S., Plumbley, M.D.: Natural conjugate gradient on complex flag manifolds for complex independent subspace analysis. In: International Conference on Artificial Neural Networks, pp. 165–174. Springer (2008)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering, 2nd edn. Springer, New York (2006)
O’Neill, B.: Semi-Riemannian Geometry, Pure and Applied Mathematics, vol. 103. Academic Press Inc, New York (1983)
Pennec, X.: Barycentric subspace analysis on manifolds. Ann. Stat. 46(6A), 2711–2746 (2018)
Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012)
Savas, B., Lim, L.H.: Quasi-Newton methods on Grassmannians and multilinear approximations of tensors. SIAM J. Sci. Comput. 32(6), 3352–3393 (2010)
Schulz, V.H.: A Riemannian view on shape optimization. Found. Comput. Math. 14(3), 483–501 (2014)
Stiefel, E.: Richtungsfelder und Fernparallelismus in n-dimensionalen Mannigfaltigkeiten. Comment. Math. Helv. 8(1), 305–353 (1935)
Tojo, K.: Totally geodesic submanifolds of naturally reductive homogeneous spaces. Tsukuba J. Math. 20(1), 181–190 (1996)
Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013)
Wang, L., Wang, X., Feng, J.: Subspace distance analysis with application to adaptive Bayesian algorithm for face recognition. Pattern Recogn. 39(3), 456–464 (2006)
Wang, R., Shan, S., Chen, X., Gao, W.: Manifold-manifold distance with application to face recognition based on image set. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8. IEEE (2008)
Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142(1–2, Ser. A), 397–434 (2013)
Acknowledgements
We thank the two anonymous referees for their exceptionally helpful suggestions and comments. KY is partially supported by National Key Research and Development Program of China No. 2018YFA0306702 and National Key Research and Development Program of China No. 2020YFA0712300, NSFC Grant No. 11801548 and NSFC Grant No. 11688101. LHL is supported by DARPA D15AP00109, HR00112190040, NSF IIS 1546413, DMS 1854831.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
KY is supported by NSFC Grant No. 11801548, NSFC Grant No. 11688101, and National Key R&D Program of China Grant No. 2018YFA0306702 and 2020YFA0712300. LHL is supported by DARPA D15AP00109 and HR00112190040, NSF IIS 1546413, DMS 1854831.
Rights and permissions
About this article
Cite this article
Ye, K., Wong, K.SW. & Lim, LH. Optimization on flag manifolds. Math. Program. 194, 621–660 (2022). https://doi.org/10.1007/s10107-021-01640-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-021-01640-3