Abstract
Many classes of methods for noisy optimization problems are based on function information computed on sequences of simplices. The Nelder-Mead, multidirectional search, and implicit filtering methods are three such methods. The performance of these methods can be explained in terms of the difference approximation of the gradient implicit in the function evaluations. Insight can be gained into choice of termination criteria, detection of failure, and design of new methods.
This research was partially supported by National Science Foundation grant #DMS–9700569.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
L. Armijo. Minimization of functions having Lipschitz-continuous first partial derivatives, Pacific J. Math., 16:1–3, 1966.
D. B. Bertsekas. On the Goldstein-Levitin-Polyak gradient projection method, IEEE Trans. Autom. Control, 21:174–184, 1976.
C. G. Broyden. Quasi-Newton methods and their application to function minimization, Math. Comp., 21:368–381, 1967.
R. H. Byrd, R. B. Schnabel and G. A. Schultz. Parallel quasi-Newton methods for unconstrained optimization, Math. Prog., 42:273–306, 1988.
J. W. David, C. Y. Cheng, T. D. Choi, C. T. Kelley and J. Gablonsky. Optimal design of high speed mechanical systems, Tech. Rep. CRSC-TR97–18, North Carolina State University, Center for Research in Scientific Computation, July 1997. Mathematical Modeling and Scientific Computing, to appear.
J. W. David, C. T. Kelley and C. Y. Cheng. Use of an implicit filtering algorithm for mechanical system parameter identification. SAE Paper 960358, 1996 SAE International Congress and Exposition Conference Proceedings, Modeling of CI and SI Engines, 189–194.
J. E. Dennis and R. B. Schnabel. Numerical Methods for Nonlinear Equations and Unconstrained Optimization, no. 16 in Classics in Applied Mathematics, SIAM, Philadelphia, 1996.
J. E. Dennis and V. Torczon. Direct search methods on parallel machines, SIAM J. Optim., 1:448–474, 1991.
A. V. Fiacco and G. P. McCormick. Nonlinear Programming, John Wiley and Sons, New York, 1968.
S. J. Fortune, D. M. Gay, B. W. Kernighan, O. Landron, R. A. Valenzuela and M. H. Wright. WISE design of indoor wireless systems, IEEE Computational Science and Engineering, Spring (1995), 58–68.
P. Gilmore. An Algorithm for Optimizing Functions with Multiple Minima, PhD thesis, North Carolina State University, Raleigh, North Carolina, 1993.
P. Gilmore and C. T. Kelley. An implicit filtering algorithm for optimization of functions with many local minima, SIAM J. Optim., 5:269–285, 1995.
P. Gilmore, C. T. Kelley, C. T. Miller and G. A. Williams. Implicit filtering and optimal design problems: Proceedings of the workshop on optimal design and control, Blacksburg VA,April 8–9, 1994, in Optimal Design and Control, J. Borggaard, J. Burkhardt, M. Gunzburger, and J. Peterson, eds., vol. 19 of Progress in Systems and Control Theory, Birkhäuser, Boston, 159176, 1995.
C. T. Kelley. Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition, Tech. Rep. CRSC-TR97–2, North Carolina State University, Center for Research in Scientific Computation, January 1997. Submitted for Publication.
J. C. Lagarias, J. A. Reeds, M. H. Wright and P. E. Wright. Convergence properties of the Nelder-Mead simplex algorithm in low dimensions, Tech. Rep. 96–4–07, AT&T Bell Laboratories, April 1996.
D. Q. Mayne and E. Polak. Nondifferential optimization via adaptive smoothing, J. Optim. Theory & Appl., 43:601–613, 1984.
K. I. M. McKinnon. Convergence of the Nelder-Mead simplex method to a non-stationary point, tech. rep., Department of Mathematics and Computer Science, University of Edinburgh, Edinburgh, 1996.
J. A. Nelder and R. Mead. A simplex method for function minimization, Comput. J., 7:308–313, 1965.
D. Stoneking, G. Bilbro, R. Trew, P. Gilmore and C. T. Kelley. Yield optimization using a GaAs process simulator coupled to a physical device model, IEEE Transactions on Microwave Theory and Techniques, 40:1353–1363, 1992.
D. E. Stoneking, G. L. Bilbro, R. J. Trew, P. Gilmore and C. T. Kelley. Yield optimization using a GaAs process simulator coupled to a physical device model, in Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, IEEE, 374–383, 1991.
V. Torczon. On the convergence of the multidimensional direct search, SIAM J. Optim., 1:123–145, 1991.
V. Torczon. On the convergence of pattern search algorithms, SIAM J. Optim., 7:1–25, 1997.
T. A. Winslow, R. J. Trew, P. Gilmore and C. T. Kelley. Doping profiles for optimum class B performance of GaAs mesfet amplifiers, in Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, IEEE, 188–197, 1991.
T. A. Winslow, R. J. Trew, P. Gilmore and C. T. Kelley. Simulated performance optimization of GaAs MESFET amplifiers, in Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, IEEE, 393–402,1991.
S. K. Zavriev. On the global optimization properties of finite-difference local descent algorithms, J. Global Optimization, 3:67–78, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Science+Business Media New York
About this chapter
Cite this chapter
Bortz, D.M., Kelley, C.T. (1998). The Simplex Gradient and Noisy Optimization Problems. In: Borggaard, J., Burns, J., Cliff, E., Schreck, S. (eds) Computational Methods for Optimal Design and Control. Progress in Systems and Control Theory, vol 24. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-1780-0_5
Download citation
DOI: https://doi.org/10.1007/978-1-4612-1780-0_5
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-1-4612-7279-3
Online ISBN: 978-1-4612-1780-0
eBook Packages: Springer Book Archive