Abstract
We consider the problem of the global minimization of a function observed with noise. This problem occurs for example when the objective function is estimated through stochastic simulations. We propose an original method for iteratively partitioning the search domain when this area is a finite union of simplexes. On each subdomain of the partition, we compute an indicator measuring if the subdomain is likely or not to contain a global minimizer. Next areas to be explored are chosen in accordance with this indicator. Confidence sets for minimizers are given. Numerical applications show empirical convergence results, and illustrate the compromise to be made between the global exploration of the search domain and the focalization around potential minimizers of the problem.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarts EHL, Laarhoven V (1985) Statistical cooling: a general approach to combinatorial optimization problems. Philips J Res 40(4):193–226
Alliot JM (1996) Techniques d’optimisation stochastique appliquées aux problèmes du contrle aérien. INPT, Habilitation Diriger des Recherches
Arora JS, Elwakeil OA, Chahande AI, Hsieh CC (1995) Global optimization methods for engineering applications: a review. Struct Optim 9:137–159
Bect J (2010) IAGO for global optimisation with noisy evaluations. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22-24 Nov. 2010. Slides available at http://www.imsv.unibe.ch/content/continuingeducation/nko_workshop/program/index_ger.html
Bellman RE (1957) Dynamic Programming. Princeton University Press, Princeton, NJ
Blum JR (1954) Multidimensional stochastic approximation methods. Ann Math Stat 25:737–744
Box GEP, Draper NR (2007) Response surfaces, mixtures, and ridge analyses. Wiley
Branke J, Meisel S, Schmidt C (2008) Simulated annealing in the presence of noise. Journal of Heuristics 14(6):627–654
Broadie M, Cicek DM, Zeevi A (2009) An adaptative multidimensional version of the kiefer-Wolfowitz stochastic. In: Rossetti MD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceeding of the 2009 winter simulation conference
De Berg M, Cheong O, van Kreveld M, Overmars M (2008) Computational geometry: algorithms and applications. Springer-Verlag
Emmerich MTM (2005) Single and multi-objective evolutionary design optimization assisted by Gaussian random field Metamodels. Dissertation zur Erlangung des Grades eines Doktors der Naturwissenschaften der Universit¨at Dortmund, Dortmund
Garcia MJ, Gonzalez CA (2004) Shape optimisation of continuum structures via evolution strategies and fixed grid finite element analysis. Struct Multidisc Optim 26:92–98
Ginsbourger D (2009) Multiples métamodèles pour l’approximation et l’optimisation de fonctions numériques multivariables. Thèse de doctorat de mathématiques appliquées, Ecole nationale supéerieure des mines de Saint-Etienne, n519MA
Gu X, Renaud JE, Batill SM, Brach RM, Budhiraja AS (2000) Worst case propagated uncertainty of multidisciplinary systems in robust design optimization. Struct Multidisc Optim 20:190–213
Horst R, Pardalos PM (1995) Handbook of global optimization. Kluwer Academic Publishers, Dordrecht Boston London
Hansen ER (1979) Global optimization using interval analysis: the one dimensional case. JOTA 29:331–344
Janusevskis J, Le Riche R (2010) Simultaneous kriging-based sampling for optimization and uncertainty propagation. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22–24 Nov 2010
Jones DR, Pertunen CD, Stuckman BE (1993) Lipschitzian optimization without the Lipschitz constant. J Optim Theory Appl 79(1):157–181
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492
Jones DR (2001) A taxonomy of global optimization methods based on response surface. J Glob Optim 21:345–383
Kiefer J, Wolfowitz J (1952) Stochastic estimation of the maximum of a regression function. Ann Math Stat 23:462–466
Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192:707–716
Kleijnen JPC, van Beers W, Van Nieuwenhuyse I (2011) Expected improvement in efficient global optimization through bootstrapped kriging. Springer, pp 1–5
Lawler EL, Wood DE (1966) Branch and Bound methods: a survey. Oper Res 14(4):699–719
Lucasius CB, Kateman G (1993) Understanding and using genetic algorithms Part 1. Concepts, properties and context. Chemometr Intell Lab Syst 19(1):1–33
Mathias K, Whitley D, Kusuma A, Stork C (1996) An empirical evaluation of genetic algorithms on noisy objective functions. CRC, pp 65–86
Nelder J, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313
Norkin V, Pflug GCh, Ruszczyński A (1996) A branch and bound method for stochastic global optimization. Math Program 83(1–3):452–450
Picheny V, Ginsourger D, Richet Y (2010) Optimization of noisy computer experiments with tunable precision. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22–24 Nov. 2010. Slides available at http://www.imsv.unibe.ch/content/continuingeducation/nko_workshop/program/index_ger.html
Robbins H, Monro S (1951) A Stochastic approximation method. Ann Math Stat 22:400–407
Rullière D, Ribereau P (2011) Information aggregation and kriging alternative in a noisy environment. Preprint. French version available on HAL
Sakata S, Ashida F (2009) Ns-kriging based microstructural optimization applied to minimizing stochastic variation of homogenized elasticity of fiber reinforced composites. Struct Multidisc Optim 38:443–453
Schonlau M (1997) Computer experiments and global optimization. PhD, Dissertation, University of Waterloo
Schubert B (1972) A sequential method seeking the global maximum of a function. SIAM J Numer Anal 9:379–388
Shin YS, Grandhi RV (2001) A global structural optimization technique using an interval method. Struct Multidisc Optim 22:351–363
Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidisc Optim 27:302–313
Smith NA, Tromble RW (2004) Sampling uniformly from the unit simplex. Technical Report, Johns Hopkins University
Strugarek C (2006) Approches variationnelles et autres contributions en optimisation stochastique. ENPC, Thèse de doctorat
Vazquez E, Villemonteix J, Sidorkiewicz M, Walter E (2008) Global optimization based on noisy evaluations: an empirical study of two statistical approaches. J Phys Conference Series 135: 012100
Villemonteix J (2009) Optimisation de fonctions coûteuses. Thèse de doctorat de physique, Université Paris Sud 11, Faculté des sciences d’Orsay, n9278
Woon SY, Querin OM, Steven GP (2001) Structural application of a shape optimization method based on a genetic algorithm. Struct Multidisc Optim 22:57–64
Acknowledgements
The authors would like to thank the anonymous reviewers and professor Ragnar Norberg for their valuable comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was presented in part at the Noisy Kriging-based Optimization (NKO) workshop, Bern, November 2010. It has been partially funded by ANR Research project ANR-08-BLAN-0314-01, by a grant from ANRT with reference 177/2008, by MIRACCLEGICC project, and Chaire BNP Paribas Cardif Management de la modélisation.
Rights and permissions
About this article
Cite this article
Rullière, D., Faleh, A., Planchet, F. et al. Exploring or reducing noise?. Struct Multidisc Optim 47, 921–936 (2013). https://doi.org/10.1007/s00158-012-0874-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-012-0874-5