Abstract
This paper considers the well-known problem of dealing with non-convexity during the minimization of a non-linear function f(x) by Newton-like methods. The proposal made here involves a curvilinear search along an approximation to the continuous steepest descent path defined by the solution of the differential equation
The algorithm we develop and describe has some features in common with trust-region methods and we present some numerical experiments in which its performance is compared with other ODE-based and trust-region methods.
Similar content being viewed by others
References
Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in linear and Non-Linear Programming. Stanford University Press (1972)
Bartholomew-Biggs, M.C., Beddiaf, S., Kane, S.J.: Traversing non-convex regions. Adv. Model. Optim. 15(2), 387–407 (2013)
Beddiaf, S.: Continuous steepest descent path for traversing non-convex regions. Adv. Model. Optim. 11(1), 3–24 (2009)
Behrman, W.: An efficient gradient flow method for unconstrained optimization. Stanford University, PhD thesis (1998)
Botsaris, C.A.: A curvilinear optimization method based upon iterative estimation of the eigensystem of the hessian matrix. J. Maths. Anal. Appl. 63(2), 396–411 (1978)
Botsaris, C.A.: Differential gradient methods. J. Maths. Anal. Appl. 63(1), 177–198 (1978)
Branch, M.A., Coleman, T.F., Li, Y.: A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput. 21(1), 1–23 (1999)
Brown, A.A.: Optimisation methods involving the solution of ordinary differential equations. PhD thesis, Hatfield Polytechnic (1986)
Brown, A.A., Bartholomew-Biggs, M.C.: Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations. J. Optim Theory Appl. 62(2), 211–224 (1989)
Broyden, C.G., Vespucci, M.T.: Krylov solvers for linear algebraic systems. Stud. Comput. Math., 11 (2004)
Byrd, R.H., Schnabel, R.B., Shultz, G.A.: Approximate solution of the trust region problem by minimization over two dimensional subspaces. Math. Program. 40, 247–263 (1988)
Christianson, B.: Global convergence using de-linked goldstein or wolfe linesearch conditions. Adv. Model. Optim. 11(1), 25–31 (2009)
Conn, A.R., Gould, N.I.M., Toint, P.T.: Trust Region Methods. MPS-SIAM Series on Optimization, Philadelphia (2000)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)
Gould, N.I.M., Orban, D., Toint, P.h.L.: Cuter and sifdec: A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)
Higham, D.J.: Trust region algorithms and timestep selection. SIAM J. Numer. Anal. 37(1), 194–210 (1999)
MATLAB: Matlab, optimization toolbox, version 7.10.0 (R2010a), www.mathworks.com. The MathWorks Inc. Natick, Massachusetts (2010)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Some results using CUTEr test problems
Appendix: Some results using CUTEr test problems
The following tables present results obtained with the methods discussed in this paper. The problems are drawn from the CUTEr test set [15] and the methods compared are the trust-region approach fminunc—denoted by TR—along wth Nimp1 and Behrman (as implemented in Algorithm 3) and the Higham method as implemented in Algorithm 4.
The stopping criteria used by all the methods were
The values used for the controlling parameters for the iterations in Algorithms 3 and 4 are as follows:
It will be noted that the tables do not report on every one of the 139 test problems. We have omitted results for problems where it is clear from the counts of iterations and function calls that no interpolation or extrapolation has taken place. This allows us to observe more clearly the differences in performance in situations that are actually relevant to our investigation.
The following notation is used in the tables:
-
n: the number of variables
Its/Fcs: the number of iterations/function calls (The maximum number of iterations allowed was 10000 and a table entry of the form 10000/10001 indicates this limit has been reached.)
-
“*”
denotes the performance on each problem that is judged “best” by the criteria discussed in Section 5.1
-
“F”
indicates a method has suffered a numerical failure such as floating overflow in a function evauation, e.g. because of an exceptionally large correction step.
Rights and permissions
About this article
Cite this article
Bartholomew-Biggs, M., Beddiaf, S. & Christianson, B. A comparison of methods for traversing regions of non-convexity in optimization problems. Numer Algor 85, 231–253 (2020). https://doi.org/10.1007/s11075-019-00811-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-019-00811-w