Abstract
Simulated annealing (SA) algorithms are capable of solving discrete and continuous problems. However, they are less efficient than other algorithms in solving applied problems because of their dependency on controlling parameters definition method. In the current work, a self-adaptive simulated annealing (SASA) method is presented based on entropy concept and thermodynamic laws in order to optimize the setting parameters. To provide a dynamic cooling rate and Markov chain length with a comparative relation to problem conditions, the proposed schedule utilizes thermodynamic concepts of entropy and ensemble average energy. In the proposed algorithm, simulation of the atomic motion is implemented based on velocity definition in thermodynamics and time definition in probabilistic processes. The SASA is evaluated by CEC2015 problem and compared with three other adaptive simulated annealing algorithms, a standard SA and four other metaheuristic methods using three different comparison criteria: Wilcoxon test, median, mean and standard deviation. The SASA has shown satisfactory outcomes in most unimodal, multimodal, and hybrid functions in CEC2015. It has proved to be more explorative, has obtained far better solutions, and has showed the best convergence speed compared with other algorithms when engaged in exploitation.
Similar content being viewed by others
Notes
\(N_{a}\) is the number of acceptances and \(N_{t}\) is the number of iterations in each inner loop.
\(\Delta T\) is the parameter for temperature reduction.
\(\sigma\left(T\right)\) standard deviation of the energy (cost function) of the points in the Markov chain at T & δ = 0.1 is called the distance parameter.
U is some upper bound on \(\left( {E\left( s \right) - E_{ \text{min} } } \right)\) and \(\gamma\) a small real number.
\(E_{ \text{min} }^{k}\) is minimal energy at current temperature, \(E_{\text{avg}}^{k}\) is average energy of attempted moves at current temperature.
\(\sigma \left( {T_{k} } \right)\) is standard deviation of the expected energy (cost function) of the points in the Markov chain at \(T_{k}\) and \(\Delta \left( {T_{k} } \right) = \sigma \left( {T_{k} } \right)/\mu\), by \(\mu \in \left[ {1,20} \right]\).
\(\alpha\) is a constant equal to temperature when change in energy is zero and \(\varGamma\) is minimum energy (cost function) achieved in any iteration + a fixed constant C.
\(U^{j}\) and \(L^{j}\) are the lower and upper bounds of the variable and \(\rho\) is a uniform random number from [0, 1].
\(\rho\) is a uniform random number from [0, 1], N is the maximal generation number, and b = 5 is a system parameter determining the degree of dependency on iteration number and \(\eta\) is randomly generated.
The number of accepted moves along the i-axis since the last step adjustment is \(n_{i}\). \(N_{\text{s}} = 20 \;{\text{and}} \;c_{i} = 2\). And r is a random number generated in the range [-1, l] \(e_{i}\) is the vector of the ith coordinate direction; \(d_{i + 1}^{j}\) is the component of the step vector d.
References
Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680. https://doi.org/10.1126/science.220.4598.671
Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092. https://doi.org/10.1063/1.1699114
Černý V (1985) Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J Optim Theory Appl 45(1):41–51. https://doi.org/10.1007/BF00940812
Rose J, Klebsch W, Wolf Jr (1990) Temperature measurement and equilibrium dynamics of simulated annealing placements. IEEE Trans Comput Aided Des Integr Circuits Syst 9(3):253–259. https://doi.org/10.1109/43.46801
Greene JW, Supowit KJ (1986) Simulated annealing without rejected moves. IEEE Trans Comput Aided Des Integr Circuits Syst 5(1):221–228. https://doi.org/10.1109/TCAD.1986.1270190
Aluffi-Pentini F, Parisi V, Zirilli F (1985) Global optimization and stochastic differential equations. J Optim Theory Appl 47(1):1–16. https://doi.org/10.1007/BF00941312
Geman S, Geman D (1984) Stochastic relaxation, gibbs distribution, and the bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell Pami-6 6:721–741
Szu H, Hartley R (1987) Fast simulated annealing. Phys Lett A 122(3–4):157–162. https://doi.org/10.1016/0375-9601(87)90796-1
Shieh H-L, Kuo C-C, Chiang C-M (2011) Modified particle swarm optimization algorithm with simulated annealing behavior and its numerical verification. Appl Math Comput 218(8):4365–4383. https://doi.org/10.1016/j.amc.2011.10.012
Siarry P, Berthiau G, Durdin F, Haussy J (1997) Enhanced simulated annealing for globally minimizing functions of many-continuous variables. ACM Trans Math Softw 23(2):209–228. https://doi.org/10.1145/264029.264043
Ali MM, Gabere MN, Zhu W (2012) A derivative-free variant called DFSA of Dekkers and Aarts’ continuous simulated annealing algorithm. Appl Math Comput 219(2):605–616. https://doi.org/10.1016/j.amc.2012.06.050
Xinchao Z (2011) Simulated annealing algorithm with adaptive neighborhood. Appl Soft Comput 11(2):1827–1836. https://doi.org/10.1016/j.asoc.2010.05.029
Xavier-De-Souza S, Suykens JAK, Vandewalle J, Ds Bolle (2010) Coupled simulated annealing. IEEE Trans Syst Man Cybern B Cybern 40(2):320–335. https://doi.org/10.1109/TSMCB.2009.2020435
Tsallis C, Da Stariolo (1996) Generalized simulated annealing. Phys A 233:395–406
Bertsimas D, Nohadani O (2010) Robust optimization with simulated annealing. J Glob Optim 48(2):323–334. https://doi.org/10.1007/s10898-009-9496-x
Wang PP, Chen D-S (1996) Continuous optimization by a variant of simulated annealing. Comput Optim Appl 6:59–71. https://doi.org/10.1007/BF00248009
Ingber L (1989) Very fast simulated re-annealing. Math Comput Model 12(8):967–973. https://doi.org/10.1016/0895-7177(89)90202-1
Dekkers A, Aarts E (1991) Global optimization and simulated annealing. Math Program 50(1–3):367–393. https://doi.org/10.1007/BF01594945
Triki E, Collette Y, Siarry P (2005) A theoretical study on the behavior of simulated annealing leading to a new cooling schedule. Eur J Oper Res 166(1):77–92. https://doi.org/10.1016/j.ejor.2004.03.035
de Vicente J, Lanchares J, Hermida R (2003) Placement by thermodynamic simulated annealing. Phys Lett A 317(5–6):415–423. https://doi.org/10.1016/j.physleta.2003.08.070
Corana A, Marchesi M, Martini C, Ridella S (1987) Minimizing multimodal functions of continuous variables with the ‘simulated annealing’ algorithm. ACM Trans Math Softw 13(3):262–280. https://doi.org/10.1145/66888.356281
Romeijn HE, Smith RL (1994) Simulated annealing for constrained global optimization. J Glob Optim 5(2):101–126. https://doi.org/10.1007/BF01100688
Salazar R, Toral R (1997) Simulated annealing using hybrid Monte Carlo. J Stat Phys 89(1):1047–1060. https://doi.org/10.1007/BF02764221
Andricioaei I, Straub J (1996) Generalized simulated annealing algorithms using Tsallis statistics: application to conformational optimization of a tetrapeptide. Phys Rev E 53(4):R3055–R3058. https://doi.org/10.1103/PhysRevE.53.R3055
Xin Y, Guojie L (1991) General simulated annealing. J Comput Sci Technol 6(4):328–329
Shojaee GK, Shakouri GH, Menhaj MB (2011) A mushy state simulated annealing. AUT J Model Simul 43(2):59–70. https://doi.org/10.22060/MISCJ.2011.174
Haines LM (1987) The application of the annealing algorithm to the construction of exact optimal designs for linear—regression models. Technometrics 29(4):439–447. https://doi.org/10.1080/00401706.1987.10488272
Lundy M, Mees A (1986) Convergence of an annealing algorithm. Math Program 34(1):111–124. https://doi.org/10.1007/BF01582166
Locatelli M (2000) Convergence of the simulated annealing algorithm for continuous global optimization. J Glob Optim 18:219–234
Bohachevsky IO, Johnson ME, Stein ML (1986) Generalized simulated annealing for function optimization. Technometrics 28(3):209–217. https://doi.org/10.1080/00401706.1986.10488128
Shen Y, Kiatsupaibul S, Zabinsky ZB, Smith RL (2007) An analytically derived cooling schedule for simulated annealing. J Glob Optim 38(3):333–365. https://doi.org/10.1007/s10898-006-9068-2
Ansari N, Sarasa R, Wang G (1993) An efficient annealing algorithm for global optimization in Boltzmann machines. Appl Intell 3(3):177–192
Zhao X, Zhou Y, Xiang Y (2019) A grouping particle swarm optimizer. Appl Intell. https://doi.org/10.1007/s10489-019-01409-4
Ning Y, Peng Z, Dai Y, Bi D, Wang J (2018) Enhanced particle swarm optimization with multi-swarm and multi-velocity for optimizing high-dimensional problems. Appl Intell 49(2):335–351. https://doi.org/10.1007/s10489-018-1258-3
Rashedi E, Nezamabadi-pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248. https://doi.org/10.1016/j.ins.2009.03.004
Tian L, Li Z, Yan X (2018) Differential evolution algorithm directed by individual difference information between generations and current individual information. Appl Intell 49(2):628–649. https://doi.org/10.1007/s10489-018-1255-6
Duan M, Yang H, Liu H, Chen J (2018) A differential evolution algorithm with dual preferred learning mutation. Appl Intell 49(2):605–627. https://doi.org/10.1007/s10489-018-1267-2
Gelfand SB, Mitter SK (1991) Simulated annealing type algorithms for multivariate optimization. Algorithmica 6:419–436
Gelfand SB, Mitter SK (1991) Weak convergence of markov and algorithms to sampling diffusions. J Optim Theory Appl 68(3):483–498
Pavlyukevich I (2007) Lévy flights, non-local search and simulated annealing. J Comput Phys 226(2):1830–1844. https://doi.org/10.1016/j.jcp.2007.06.008
Reif F, Sevenich RA (1971) Statistical physics (Berkeley physics course—volume 5). Am J Phys 39(2):235. https://doi.org/10.1119/1.1986109
Szu HH, Hartley RL (1987) Nonconvex optimization by fast simulated annealing. Proc IEEE 75(11):1538–1540. https://doi.org/10.1109/PROC.1987.13916
Liang JJ, Qu BY, Suganthan PN, Chen Q (2014) Problem definitions and evaluation criteria for the CEC 2015 competition on learning-based real-parameter single objective optimization. Technical report, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Technical Report, Nanyang Technological University, Singapore
Arora J, Elwakeil O, Chahande A, Hsieh C (1995) Global optimization methods for engineering applications: a review. Struct Optim 9(3–4):137–159
Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657. https://doi.org/10.1109/TEVC.2006.872133
Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195. https://doi.org/10.1162/106365601750190398
Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295. https://doi.org/10.1109/tevc.2005.857610
Garcia-Martinez C, Lozano M, Herrera F, Molina D, Sanchez AM (2008) Global and local real-coded genetic algorithms based on parent-centric crossover operators. Eur J Oper Res 185(3):1088–1113. https://doi.org/10.1016/j.ejor.2006.06.043
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shojaee Ghandeshtani, K., Mashhadi, H.R. An entropy-based self-adaptive simulated annealing. Engineering with Computers 37, 1329–1355 (2021). https://doi.org/10.1007/s00366-019-00887-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00366-019-00887-x