Abstract
Many constrained optimization algorithms use a basis for the null space of the matrix of constraint gradients. Recently, methods have been proposed that enable this null space basis to vary continuously as a function of the iterates in a neighborhood of the solution. This paper reports results from topology showing that, in general, there is no continuous function that generates the null space basis of all full rank rectangular matrices of a fixed size. Thus constrained optimization algorithms cannot assume an everywhere continuous null space basis. We also give some indication of where these discontinuities must occur. We then propose an alternative implementation of a class of constrained optimization algorithms that uses approximations to the reduced Hessian of the Lagrangian but is independent of the choice of null space basis. This approach obviates the need for a continuously varying null space basis.
Similar content being viewed by others
References
J.F. Adams, “Vector fields on spheres“,Annals of Mathematics 75 (1962) 603–632.
T.F. Coleman and A.R. Conn, “On the local convergence of a quasi-Newton method for a nonlinear programming problem“,SIAM Journal on Numerical Analysis 21 (1984) 755–769.
T.F. Coleman and D.C. Sorensen, “A note on the computation of an orthonormal basis for the null space of a matrix“,Mathematical Programming 29 (1984) 234–242.
B. Eckmann, “Stetige Lösungen linearer Gleichungssysteme“,Commentarii Mathematici Helvetici 15 (1942) 318–339.
P.E. Gill, W. Murray, M.A. Saunders, G.W. Stewart and M.H. Wright, “Properties of a representation of a basis for the null space“,Mathematical Programming 33 (1985) 172–186.
P.E. Gill, W. Murray, M.A. Saunders and M.H. Wright, “On the representation of a basis for the null space”, Report SOL 83-19, Stanford University (Stanford, California, 1983).
J. Goodman, “Newton's Method for Constrained Optimization“,Mathematical Programming 33 (1985) 162–171.
P.J. Hilton and S. Wylie,Homology theory (Cambridge University Press, Cambridge, 1962).
G.P. McCormick, “Locating an isolated global minimizer of a constrained nonconvex program“,Mathematics of Operations Research 5 (1980) 435–443.
J. Nocedal and M. Overton, “Projected Hessian updating algorithms for nonlinearly constrained optimization“,SIAM Journal on Numerical Analysis 22 (1985) 821–850.
T. Wajewski, “Sur les matrices dont les éléments sont des fonctions continués,Compositio Mathematica 2 (1935) 63–68.
G.W. Whitehead, “Note on cross-sections in Stiefel manifolds“,Commentarii Mathematici Helvetici 37 (1963) 239–240.
R.S. Womersley and R. Fletcher, “An algorithm for composite nonsmooth optimization problems”, Department of Mathematics Report NA/60, University of Dundee (Dundee, 1982).
Author information
Authors and Affiliations
Additional information
Research supported by NSF grant MCS 81-15475 and DCR-8403483
Research supported by ARO contracts DAAG 29-81-K-0108 and DAAG 29-84-K-0140
Rights and permissions
About this article
Cite this article
Byrd, R.H., Schnabel, R.B. Continuity of the null space basis and constrained optimization. Mathematical Programming 35, 32–41 (1986). https://doi.org/10.1007/BF01589439
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01589439