Abstract
It is well known that the Newton method has a second order rate of convergence and that it is widely used to solve optimization problems and nonlinear equations which arise from computational science, engineering analysis and other applications. However, two big disadvantages hinder its application: high computational cost for large scale problems and poor global performance in some complicated and difficult problems. Some inexact Newton methods have emerged over time. Among them, the Newton preconditioned conjugate gradient method is the most efficient and popular approach to overcome the first shortcoming while keeping rapid convergence. In this paper, we have improved the global performance of the inexact Newton method by developing a nonmonotone line search technique. We have also proved the global convergence of the proposed method under some conditions. Numerical experiments on a set of standard test problems are reported. They have shown that the proposed algorithm is promising.
Similar content being viewed by others
References
Ahookhosh, M., Ghaderi, S.: On efficiency of nonmonotone Armijo-type line searches. arXiv preprint arXiv:1408.2675, (2014)
Amini, K., Ahookhosh, M., Nosratipour, H.: An inexact line search approach using modified nonmonotone strategy for unconstrained optimization. Numer. Algorithms 66(1), 49–78 (2014)
Dai, Y.H., Zhang, H.C.: Adaptive two-point stepsize gradient algorithm. Numer. Algorithms 27(4), 377–385 (2001)
Dai, Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112(2), 315–330 (2002)
Dembo, R.S., Eisenstat, S.C., Steihaug, T.: Inexact Newton method. SIAM J. Numer. Anal. 19(2), 400–408 (1982)
Dembo, R.S., Steihaug, T.: Truncated-Newton algorithms for large-scale unconstrained optimization. Math. Program. 26(2), 190–212 (1983)
Deng, N.Y., Xiao, Y., Zhou, F.J.: Nonmonotonic trust-region algorithm. J. Optim. Theory Appl. 76(2), 259–285 (1993)
Deng, N.Y., Wang, Z.Z.: Theoretical efficiency of an inexact Newton method. J. Optim. Theory Appl. 105(1), 97–112 (2000)
Gao, H., Tong, X.J., Dai, Y.H.: Barzilai-Borwein-like methods for the extreme eigenvalue problem. J. Ind. Manag. Optim. 11(3), 999–1019 (2015)
Golub, G.H., Van Loan, C.F.: Matrix Computations. JHU Press, Baltimore (2012)
Gould, N.I.M., Orban, D., Toint, P.L.: CUTEr and SifDec: A constrained and unconstrained testing environment, revisited. ACM T. Math. Software(TOMS). 29(4), 373–394 (2003)
Grippo, L., Lamparillo, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23(4), 707–716 (1986)
Grippo, L., Lamparillo, F., Lucidi, S.: Newton-type algorithms with nonmonotone line search for large-scale unconstrained optimization. Springer, Berlin (1988)
Grippo, L., Lamparillo, F., Lucidi, S.: A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60(3), 401–419 (1989)
Grippo, L., Lamparillo, F., Lucidi, S.: A class of nonmonotone stabilization methods in unconstrained optimization. Numer. Math. 59(1), 779–805 (1991)
Jiang, B., Dai, Y.H.: Feasible Barzilai-Borwein-like methods for extreme symmetric eigenvalue problems. Optim. Method Softw. 28(4), 756–784 (2013)
Liu, G., Han, J., Sun, D.: Global convergece of the BFGS algorithm with nonmonotone linesearch. Optimization 34(2), 147–159 (1995)
Lu, Z.S., Xiao, L.: Randomized block coordinate non-monotone gradient method for a class of nonlinear programming. (2013). arXiv:1306.5918
Saad, Y.: Iterative methods for sparse linear systems. SIAM (2003)
Toint, P.L.: Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints. Math. Program. 77(3), 69–94 (1997)
Toint, P.L.: Towards an efficient sparsity exploiting Newton method for minimization. Sparse matrices and their uses. Academic Press, London (1981)
Zhang, H.B., Deng, N.Y.: An improved inexact Newton method. J. Glob. Optim. 39(2), 221–234 (2007)
Zhu, D.: Nonmonotone backtracking inexact quasi-Newton algorithms for solving smooth nonlinear equations. Appl. Math. Comput. 161(3), 875–895 (2005)
Acknowledgments
The authors would like to thank Prof. Yu-Hong Dai from Chinese Academy of Sciences for discussion on this paper and careful checking on an early version of this manuscript, and they would like to thank Piaoyang Qi for his help on the numerical experiments. The authors also thank the anonymous referees very for their patient and valuable comments, which improved the quality of this paper greatly. This work was supported by National Natural Science Foundation of China (Grant No. 61179033), by Collaborative Innovation Center on Beijing Society-building and Social Governance and by China Postdoctoral Science Foundation (No. 2014M561081)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gao, H., Zhang, HB., Li, ZB. et al. A nonmonotone inexact Newton method for unconstrained optimization. Optim Lett 11, 947–965 (2017). https://doi.org/10.1007/s11590-015-0976-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-015-0976-2