Abstract
We present a hybrid algorithm that combines a genetic algorithm with the Barzilai–Borwein gradient method. Under specific assumptions the new method guarantees the convergence to a stationary point of a continuously differentiable function, from any arbitrary initial point. Our preliminary numerical results indicate that the new methodology finds efficiently and frequently the global minimum, in comparison with the globalized Barzilai–Borwein method and the genetic algorithm of the Toolbox of Genetic Algorithms of MatLab.
Similar content being viewed by others
References
Azofeifa D., Clark N., Vargas W.: Optical and electrical properties of terbium films as a function of hydrogen concentration. Phys. Stat. B – Basic Solid State Phys. 242, 2005–2009 (2005)
Barzilai J., Borwein J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)
Bellavia S., Morini B.: A globally convergent Newton-GMRES subspace method for systems of nonlinear equations. SIAM J. Sci. Comput. 23, 940–960 (2001)
Bello L., Raydan M.: Convex constrained optimization for the seismic reflection tomography problem. J. Appl. Geophys. 62, 158–166 (2007)
Bielschowsky R.H., Friedlander A., Gomes F.A., Martínez J.M., Raydan M.: An adaptive algorithm for bound constrained quadratic minimization. Invest. Oper. 7, 67–102 (1997)
Birgin E.G., Chambouleyron I., Martínez J.M.: Estimation of the optical constants and the thickness of thin films using unconstrained optimization. J. Comput. Phys. 151, 862–880 (1999)
Birgin E.G., Chambouleyron I., Martínez J.M.: Optimization problems in the estimation of parameters of thin films and the elimination of the influence of the substrate. J. Comput. Appl. Math. 152, 35–50 (2003)
Birgin E.G., Evtushenko Y.G.: Automatic differentiation and spectral projected gradient methods for optimal control problems. Optim. Methods Softw. 10, 125–146 (1998)
Birgin E.G., Martínez J.M., Mascarenhas W.F., Ronconi D.P.: Method of sentinels for packing items within arbitrary convex regions. J. Oper. Res. Soc. 57, 735–746 (2006)
Birgin E.G., Martínez J.M., Nishihara F.H., Ronconi D.P.: Orthogonal packing of rectangular items within arbitrary convex regions by nonlinear optimization. Comput. Oper. Res. 33, 3535–3548 (2006)
Birgin E.G., Martínez J.M., Raydan M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10, 1196–1211 (2000)
Birgin E.G., Martínez J.M., Raydan M.: Algorithm 813: SPG—software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001)
Birgin E.G., Martínez J.M., Raydan M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23, 539–559 (2003)
Birgin E.G., Martínez J.M., Ronconi D.P.: Minimization subproblems and heuristics for an applied clustering problem. Eur. J. Oper. Res. 146, 19–34 (2003)
Birgin E.G., Martínez J.M., Ronconi D.P.: Optimizing the packing of cylinders into a rectangular container: a nonlinear approach. Eur. J. Oper. Res. 160, 19–33 (2005)
Brown P.N., Saad Y.: Hybrid Krylov methods for nonlinear systems of equations. SIAM J. Sci. Comput. 11, 450–481 (1990)
Brown P.N., Saad Y.: Convergence theory of nonlinear Newton–Krylov algorithms. SIAM J. Optim. 4, 297–330 (1994)
Castillo Z., Cores D., Raydan M.: Low cost optimization techniques for solving the nonlinear seismic reflection tomography problem. Optim. Eng. 1, 155–169 (2000)
Coleman T.F., Li Y.: On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Program. 67, 189–224 (1994)
Coleman T.F., Li Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)
Cores D., Fung G.M., Michelena R.J.: A fast and global tow point low storage optimization technique for tracing rays in 2D and 3D isotropic media. J. Appl. Geophys. 45, 273–287 (2000)
Cores D., Loreto M.: A generalized two point ellipsoidal anisotropic ray tracing for converted waves. Optim. Eng. 8, 373–396 (2007)
Curiel F., Vargas W.E., Barrera R.G.: Visible spectral dependence of the scattering and absorption coefficients of pigmented coatings from inversion of diffuse reflectance spectral. Appl. Opt. 41, 5969–5978 (2002)
Dai Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315–330 (2002)
Dai Y.H.: Alternate step gradient method. Optimization 52, 395–415 (2003)
Dai Y.H., Fletcher R.: On the asymptotic behaviour of same new gradient methods. Math. Program. 103, 541–559 (2005)
Dai Y.H., Fletcher R.: Project Barzilai–Borwein methods for large-scale box-constrained quadratic programming. Numerische Mathematik 100, 21–47 (2005)
Dai Y.H., Hager W., Schittkowski K., Zhang H.C.: The cyclic Barzilai–Borwein method for unconstrained optimization. IMA J. Numer. Anal. 26, 604–627 (2006)
Dai Y.H., Liao L.Z.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1–10 (2002)
Dai Y.H., Yuan J.Y., Yuan Y.X.: Modified two-point stepsize gradient methods for unconstrained optimization. Comput. Optim. Appl. 22, 103–109 (2002)
Dai Y.H., Zhang H.C.: Adaptive two-point stepsize gradient algorithm. Numer. Algorithms 27, 377–385 (2001)
Deidda G.P., Bonomi E., Manzi C.: Inversion of electrical conductivity data with tikhonov regularization approach: some considerations. Ann. Geophys. 46, 549–558 (2003)
Dennis J.E., Schnabel R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewoog Cliffs (1983)
Davis L.: The Handbook of Genetic Algorithms. Van Nostrand Reingold, New York (1991)
Fletcher R.: Low storage methods for unconstrained optimization. Lectures in Applied Mathematics (AMS) 26, 165–179 (1990)
Fletcher R.: On the Barzilai–Borwein method. Appl. Optim. 96, 235–236 (2006)
Francisco J.B., Martínez J.M., Martínez L.: Density-based globally convergent trust-region methods for self-consistent field electronic structure calculations. J. Math. Chem. 40, 349–377 (2006)
Friedlander A., Martínez J.M., Molina B., Raydan M.: Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36, 275–289 (1999)
Friedlander A., Martínez J.M., Raydan M.: A new method for large-scale constrained convex quadratic minimization problems. Optim. Methods Softw. 5, 57–74 (1995)
Glunt W., Hayden T.L., Raydan M.: Molecular conformation from distance matrices. J. Comput. Chem. 14, 114–120 (1993)
Goldberg D.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Readomg (1989)
Hendrickson B.A.: The molecule problem: exploiting structure in global optimization. SIAM J. Optim. 5, 835–857 (1995)
Holland J.: Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor (1975)
James C.S.: Introduction to Stochastics Search and Optimization. Wiley-Interscience, New Jersey (2003)
La Cruz, W.: Derivative-free residual algorithm for solving weakly nonlinear equations. IMA J. Numer. Anal. (2007), Submitted
La Cruz W., Martínez J.M., Raydan M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math. Comput. 75, 1449–1466 (2006)
La Cruz W., Raydan M.: Nonmonotone spectral methods for large-scale nonlinear systems. Optim. Methods Softw. 18, 583–599 (2003)
Meza J.C., Martinez M.L.: On the use of direct search methods for the molecular conformation problem. J. Comput. Chem. 15, 627–632 (1994)
Michalewicz Z.: Genetic Algorithms + Data Structures = Evolution Programs. AI Series. Springer- Verlag, New York (1994)
Molina B., Raydan M.: Preconditioned Barzilai–Borwein method for the numerical solution of partial differential equations. Numer. Algorithms 13, 45–60 (1996)
Nelder J.A., Mead R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)
Ramirez-Porras A., Vargas-Castro E.: Transmission of visible light through oxidized copper films: feasibility of using a spectral projected gradient method. Appl. Opt. 43, 1508–1514 (2004)
Raydan M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321–326 (1993)
Raydan M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)
Serafini T., Zanghirati G., Zanni T.: Gradient projection methods for quadratic programs and applications in training support vector machines. Optim. Methods Softw. 20, 353–378 (2005)
Vargas W.E.: Inversion methods from Kubelka–Munk analysis. J. Opt. A - Pure Appl. Opt. 4, 452–456 (2002)
Vargas W.E., Azofeifa D.E., Clark N.: Retrieved optical properties of thin films on absorbing substrates from transmittance measurements by application of a spectral projected gradient method. Thin Solid Films 425, 1–8 (2003)
Wells C., Glunt W., Hayden T.L.: Searching conformational space with the spectral distance geometry algorithm. J. Mol. Struct. (Theochem) 308, 263–271 (1994)
Wright A.H.: Genetic algorithms for real parameter optimization. In: Rawlins J.E. (ed.) Foundations of Genetic Algorithms, pp. 205–218. Morgan Kaufmann (1991)
Zeev N., Savasta O., Cores D.: Non-monotone spectral projected gradient method applied to full waveform inversion. Geophys. Prospect. 54, 525–534 (2006)
Zhina J.: Applications of conditional nonlinear optimal perturbation to the study of the stability and sensitivity of the Jovian atmosphere. Adv. Atmos. Sci. 23, 775–783 (2006)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
La Cruz, W., Noguera, G. Hybrid spectral gradient method for the unconstrained minimization problem. J Glob Optim 44, 193–212 (2009). https://doi.org/10.1007/s10898-008-9318-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-008-9318-6