Skip to main content
SpringerLink
Log in

Trends in Consensus-Based Optimization

  • Chapter
  • First Online:
Active Particles, Volume 3

Abstract

In this chapter we give an overview of the consensus-based global optimization algorithm and its recent variants. We recall the formulation and analytical results of the original model, and then we discuss variants using component-wise independent or common noise. In combination with mini-batch approaches those variants were tailored for machine learning applications. Moreover, it turns out that the analytical estimates are dimension independent, which is useful for high-dimensional problems. We discuss the relationship of consensus-based optimization with particle swarm optimization, a method widely used in the engineering community. Then we survey a variant of consensus-based optimization that is proposed for global optimization problems constrained to hyper-surfaces. We conclude the chapter with remarks on applications, preprints and open problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mohan, B. C. and Baskaran, R.: A survey: Ant colony optimization-based recent research and implementation on several engineering domain. Expert Syst. Appl. 39, 4618–4627 (2012)

    Article  Google Scholar 

  2. Karaboga, D., Gorkemli, B., Ozturk, C., and Karaboga, N.: A comprehensive survey: Artificial bee colony (ABC) algorithm and applications. Artif. Intell. Rev. 42, 21–57 (2014)

    Article  Google Scholar 

  3. Yang, X.-S.: Firefly Algorithms for Multimodal Optimization. In: Stochastic Algorithms: Foundations and Applications, SAGA 2009, Lecture Notes in Computer Sciences, Vol. 5792, pp. 169–178 (2009)

    Article  Google Scholar 

  4. Jamil, M. and Yang, X.-S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4, 150–194 (2013)

    MATH  Google Scholar 

  5. Bayraktar, Z., Komurcu, M. Bossard, J.A. and Werner, D.H.: The Wind Driven Optimization Technique and its Application in Electromagnetics. In: IEEE Transactions on Antennas and Propagation 61, 2745–2757 (2013)

    MathSciNet  MATH  Google Scholar 

  6. Henderson, D., Jacobson, S.H. and Johnson, A.W.: The theory and practice of simulated annealing. In: Handbook of Metaheuristics, International Series in Operations Research & Management Science, 57 287–319, Springer, Boston (2003)

    Google Scholar 

  7. Monmarché, P.: Hypocoercivity in metastable settings and kinetic simulated annealing, Probab. Theory Relat. Fields 172,1215–1248 (2018)

    Article  MathSciNet  Google Scholar 

  8. Chak, M., Kantas, N. and Pavliotis, G.A.: On the Generalised Langevin Equation for Simulated Annealing, arXiv:2003.06448v3 (2021)

    Google Scholar 

  9. Kennedy, J. and Eberhart, R.C.: Particle Swarm Optimization. Proc. IEEE Int. Conf. Neu. Net. 4, 1942–1948 (1995)

    Google Scholar 

  10. Hegselmann, R. and Krause, U.: Opinion dynamics and bounded confidence models, analysis, and simulation. J. Artif. Soc. Social Simulat. 5, 1–33 (2002)

    Google Scholar 

  11. Pinnau, R., Totzeck, C., Tse, O. and Martin, S.: A consensus-based model for global optimization and its mean-field limit. Math. Meth. Mod. Appl. Sci. 27,183–204 (2017)

    Article  MathSciNet  Google Scholar 

  12. Carrillo, J.A., Choi, Y.-P., Totzeck, C. and Tse, O.: An analytical framework for a consensus-based global optimization method. Math. Mod. Meth. Appl. Sci. 28, 1037–1066 (2018)

    Article  MathSciNet  Google Scholar 

  13. Carrillo, J.A., Jin, S., Li, L. and Zhu, Y.: A consensus-based global optimization method for high dimensional machine learning problems. ESAIM: COCV 27, S5 (2021)

    MathSciNet  MATH  Google Scholar 

  14. Ha, S.-Y. and Jin, S. and Kim, D.: Convergence of a first-order consensus-based global optimization algorithm. Math. Mod. Meth. Appl. Sci. 30, 2417–2444 (2020)

    Article  MathSciNet  Google Scholar 

  15. Totzeck, C and Wolfram, M.-T.: Consensus-Based Global Optimization with Personal Best. Math. Biosci. Eng. 17, 6026–6044 (2020)

    Article  MathSciNet  Google Scholar 

  16. Fornasier, M. and Huang, H. and Pareschi, L. and Sünnen, P.: Consensus-based optimization on hypersurfaces: well-posedness and mean-field limit. Math. Mod. Meth. Appl. Sci. 30, 2725–2751 (2020)

    Article  MathSciNet  Google Scholar 

  17. Dembo, A. and Zeitouni, O.: Large Deviations Techniques and Applications, Applications of Mathematics Vol. 38, Springer Science and Business Media (2009)

    Google Scholar 

  18. Higham, D.J.: An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations. SIAM Rev. 43, 525–546 (2001)

    Article  MathSciNet  Google Scholar 

  19. Robbins, H. and Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)

    Article  MathSciNet  Google Scholar 

  20. Jin, S. and Li, L., Liu, J.-G.: Random batch methods (RBM) for interacting particle systems. J. Comput. Phys. 400, 108877 (2020)

    Article  MathSciNet  Google Scholar 

  21. Ha, S.-Y. and Jin, S. and Kim, D.: Convergence and error estimates for time-discrete consensus-based optimization algorithms. Numer. Math. 147, 255–282 (2021)

    Article  MathSciNet  Google Scholar 

  22. Poli, R. and Kennedy, J. and Blackwell, T.: Particle swarm optimization - An overview. Swarm Intell. 1, 33–57 (2007)

    Article  Google Scholar 

  23. Grassi, S. and Pareschi, L.: From particle swarm optimization to consensus based optimization: stochastic modeling and mean-field limit. arXiv:2012.05613 (2020)

    Google Scholar 

  24. Fornasier, M. and Huang, H. and Pareschi, L. and Sünnen, P.: Consensus-based optimization on hypersurfaces: well-posedness and mean-field limit. arXiv:2001.11988 (2020)

    Google Scholar 

  25. Sznitman, A.-S.: Topics in propagation of chaos. In: Ecole d’été de probabilités de Saint-Flour XIX – 2089, 165–251. Springer (1991)

    Google Scholar 

  26. Totzeck, C., Pinnau, R., Blauth, S. and Schotthöfer, S.: A Numerical Comparison of Consensus-Based Global Optimization to other Particle-based Global Optimization Schemes. PAMM 18, e201800291 (2018)

    Article  Google Scholar 

  27. Golse, F.: On the Dynamics of Large Particle Systems in the Mean Field Limit. In: Macroscopic and Large Scale Phenomena: Coarse Graining, Mean Field Limits and Ergodicity, pp. 1–144. Springer (2016)

    Google Scholar 

  28. Chen, J., Jin, S. and Lyu, L.: A Consensus-based global optimization method with adaptive momentum estimation, arXiv:2012.04827 (2020)

    Google Scholar 

  29. Huang, H. and Qiu, J.: On the mean-field limit for the consensus-based optimization. arXiv:2105.12919v1 (2021)

    Google Scholar 

  30. Fornasier, M., Klock, T. and Riedl, K.: Consensus-based optimization methods converge globally in mean-field law, arXiv:2103.15130v3 (2021)

    Google Scholar 

  31. Carrillo, J.A., Hoffmann, F., Stuart, A.M. and Vaes, U.: Consensus Based Sampling. arXiv:2106.02519v1 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Natural Sciences, University of Wuppertal, Wuppertal, Germany

    Claudia Totzeck

Authors
  1. Claudia Totzeck
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudia Totzeck .

Editor information

Editors and Affiliations

  1. Department of Mathematical Sciences, Politecnico di Torino, Turin, Italy

    Nicola Bellomo

  2. Mathematical Institute, University of Oxford, Oxford, UK

    José Antonio Carrillo

  3. Department of Mathematics and Institute for Physical Science & Technology, University of Maryland, College Park, MD, USA

    Eitan Tadmor

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Totzeck, C. (2022). Trends in Consensus-Based Optimization. In: Bellomo, N., Carrillo, J.A., Tadmor, E. (eds) Active Particles, Volume 3. Modeling and Simulation in Science, Engineering and Technology. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-93302-9_6

Download citation

Publish with us

Policies and ethics

Search

Navigation