Abstract
Sequential quadratic programming (SQP) methods are known to be efficient for solving a series of related nonlinear optimization problems because of desirable hot and warm start properties—a solution for one problem is a good estimate of the solution of the next. However, standard SQP solvers contain elements to enforce global convergence that can interfere with the potential to take advantage of these theoretical local properties in full. We present two new predictor–corrector procedures for solving a nonlinear program given a sufficiently accurate estimate of the solution of a similar problem. The procedures attempt to trace a homotopy path between solutions of the two problems, staying within the local domain of convergence for the series of problems generated. We provide theoretical convergence and tracking results, as well as some numerical results demonstrating the robustness and performance of the methods.
Similar content being viewed by others
References
Allgower, E.L., Georg, K.: Introduction to Numerical Continuation Methods. Colorado State University Press, Fort Collins (1990)
Bonnans, J.: Local analysis of Newton-type methods for variational inequalities and nonlinear programming. Appl. Math. Optim. 29, 161–186 (1994)
Bonnans, J.F., Shapiro, A.: Optimization problems with perturbations: a guided tour. SIAM Rev. 40, 228–264 (1998)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, Berlin (2000)
Diehl, M.: Real-time optimization for large scale nonlinear processes. PhD thesis, Universität Heidelberg (2001)
Facchinei, F., Fischer, A., Herrich, M.: A family of Newton methods for nonsmooth constrained systems with nonisolated solutions. Math. Methods Oper. Res. 1–11 (2011)
Facchinei, F., Fischer, A., Kanzow, C.: On the accurate identification of active constraints. SIAM J. Optim. 9, 14–32 (1998)
Fernández, D., Solodov, M.: Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems. Math. Program. 125, 47–73 (2010)
Ferreau, H.: An online active set strategy for fast solution of parametric quadratic programs with applications to predictive engine control. Master’s thesis, University of Heidelberg (2006)
Ferreau, H.: qpOASES - An open-source implementation of the online active set strategy for fast model predictive control. In: Proceedings of the Workshop on Nonlinear Model Based Control: Software and Applications, Loughborough, pp. 29–30 (2007)
Ferreau, H., Kirches, C., Potschka, A., Bock, H., Diehl, M.: qpOASES: A parametric active-set algorithm for quadratic programming. Math. Program. Comput. 1–37 (2013)
Gal, T.: A historical sketch on sensitivity analysis and parametric programming. In: Gal, T., Greenberg, H. (eds.) Advances in Sensitivity Analysis and Parametric Programming. International series in operations research and management science, vol. 6, pp. 1–10. Springer, New York (1997)
Gfrerer, H., Guddat, J., Wacker, H.: A globally convergent algorithm based on imbedding and parametric optimization. Computing 30(3), 225–252 (1983)
Gill, P., Kungurtsev, V., Robinson, D.: A regularized SQP method convergent to second order optimal points. Technical Report 13–04, UCSD CCoM (2013)
Gould, N., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads. Cahier du GERAD G 2013, 27 (2013)
Guddat, J., Vasquez, F.G., Jongen, H.: Parametric Optimization: Singularities. Pathfollowing and Jumps. Teubner, Stuttgart (1990)
Guddat, J., Wacker, H., Zulehner, W.: On imbedding and parametric optimizationa concept of a globally convergent algorithm for nonlinear optimization problems. In: Fiacco, A. (ed.) Sensitivity. Stability and parametric analysis, volume 21 of mathematical programming studies, pp. 79–96. Springer, Berlin (1984)
Hager, W.W., Gowda, M.S.: Stability in the presence of degeneracy and error estimation. Math. Program. 85(1), 181–192 (1999)
Hock, W., Schittkowski, K.: Lecture Notes in Economics and Mathematical Systems. Test examples for nonlinear programming codes. Springer, Berlin (1981)
Izmailov, A.: Solution sensitivity for Karush–Kuhn–Tucker systems with non-unique Lagrange multipliers. Optimization 59(5), 747–775 (2010)
Izmailov, A.F., Solodov, M.V.: Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)
Johnson, T.C., Kirches, C., Wächter. A.: An active-set quadratic programming method based on sequential hot-starts. 2013. Available at optimization online.
Jongen, H.T., Jonker, P., Twilt, F.: Critical sets in parametric optimization. Math. Program. 34(3), 333–353 (1986)
Jongen, H.T., Weber, G.W.: On parametric nonlinear programming. Ann. Oper. Res. 27, 253–283 (1990)
Klatte, D., Kummer, B.: Stability properties of infima and optimal solutions of parametric optimization problems. In: Demyanov, V., Pallaschke, D. (eds.) Nondifferentiable Optimization: Motivations and Applications. Lecture notes in economics and mathematical systems, vol. 255, pp. 215–229. Springer, Berlin (1985)
Kojima, M.: Strongly stable stationary solutions in nonlinear programs. In: Robinson, S. M. (ed.) Analysis and Computation of Fixed Points, pp. 93–138. Academic Press, New York (1980)
Kungurtsev, V.: Second Derivative SQP Methods. PhD thesis, UC-San Diego (2013)
Kyparisis, J.: On uniqueness of Kuhn–Tucker multipliers in nonlinear programming. Math. Program. 32(2), 242–246 (1985)
Levy, A.B.: Solution sensitivity from general principles. SIAM J. Control Optim. 40, 1–38 (2001)
Levy, A.B., Rockafellar, R.: Advances in Nonsmooth Optimization. Sensitivity of solutions in nonlinear programs with nonunique multipliers. World Scientific Publishing, Singapore (1995)
Lundberg, B.N., Poore, A.B.: Numerical continuation and singularity detection methods for parametric nonlinear programming. SIAM J. Optim. 3, 134–154 (1993)
Mostafa, E.-S.M., Vicente, L.N., Wright, S.J.: Global Optimization and Constraint Satisfaction. Numerical behavior of a stabilized SQP method for degenerate NLP problems, pp. 123–141. Springer, Berlin (2003)
Poore, A., Tiahrt, C.: Bifurcation problems in nonlinear parametric programming. Math. Program. 39, 189–205 (1987)
Ralph, D., Dempe, S.: Directional derivatives of the solution of a parametric nonlinear program. Math. Program. 70, 159–172 (1995)
Robinson, S.: Perturbed Kuhn–Tucker points and rates of convergence for a class of nonlinear programming algorithms. Math. Program. 7, 1–16 (1974)
Robinson, S.M.: Stability theory for systems of inequalities. Part I: linear systems. SIAM J. Numer. Anal. 12, 754–769 (1975)
Robinson, S.M.: Stability theory for systems of inequalities, Part II: differentiable nonlinear systems. SIAM J. Numer. Anal. 13(4), 497–513 (1976)
Robinson, S.M.: Optimality and Stability in Mathematical Programming. Generalized equations and their solutions, part II: applications to nonlinear programming. Springer, Berlin (1982)
Sequeira, S., Graellis, M., Puigjaner, L.: Real-time evolution for on-line optimization of continuous processes. Ind. Eng. Chem. Res. 41, 1815–1825 (2002)
Seydel, R.: Practical Bifurcation and Stability Analysis. Springer, New York (2010)
Tran-Dinh, Q., Savorgnan, C., Diehl, M.: Adjoint-based predictor–corrector sequential convex programming for parametric nonlinear optimization. SIAM J. Optim. 22(4), 12581284 (2012)
Wachsmuth, G.: On LICQ and the uniqueness of Lagrange multipliers. Oper. Res. Lett. 41(1), 78–80 (2013)
Watson, L.T.: Solving the nonlinear complementarity problem by a homotopy method. SIAM J. Control Optim. 17, 36–46 (1979)
Wright, S.J.: Superlinear convergence of a stabilized SQP method to a degenerate solution. Comput. Optim. Appl. 11, 253–275 (1998)
Wright, S.J.: An algorithm for degenerate nonlinear programming with rapid local convergence. SIAM J. Optim. 15(3), 673–696 (2005)
Zavala, V., Anitescu, M.: Real-time nonlinear optimization as a generalized equation. SIAM J. Control Optim. 48(8), 5444–5467 (2010)
Acknowledgments
We would like to thank Greg Horn for insightful discussions in regards to some of the details of the numerical software implementation of the algorithms, in particular with regards to parallelization. In addition, we’d like to thank the reviewers for their helpful suggestions for improving the presentation of the material in the paper. This research was supported by Research Council KUL: PFV/10/002 Optimization in Engineering Center OPTEC, GOA/10/09 MaNet and GOA/10/11 Global real-time optimal control of autonomous robots and mechatronic systems. Flemish Government: IOF/KP/SCORES4CHEM, FWO: PhD/postdoc Grants and projects: G.0320.08 (convex MPC), G.0377.09 (Mechatronics MPC); IWT: PhD Grants, projects: SBO LeCoPro; Belgian Federal Science Policy Office: IUAP P7 (DYSCO, Dynamical systems, control and optimization, 2012-2017); EU: FP7-EMBOCON (ICT- 248940), FP7-SADCO ( MC ITN-264735), FP7-TEMPO, ERC ST HIGHWIND (259 166), Eurostars SMART, ACCM.
It was also supported by the European social fund within the framework of realizing the project “Support of inter-sectoral mobility and quality enhancement of research teams at the Czech Technical University in Prague”, CZ.1.07/2.3.00/30.0034.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kungurtsev, V., Diehl, M. Sequential quadratic programming methods for parametric nonlinear optimization. Comput Optim Appl 59, 475–509 (2014). https://doi.org/10.1007/s10589-014-9696-2
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-014-9696-2
Keywords
- Parametric nonlinear programming
- Nonlinear programming
- Nonlinear constraints
- Sequential quadratic programming
- SQP methods
- Stabilized SQP
- Regularized methods
- Model predictive control