Abstract
We introduce a generalized version of the Metropolis-adjusted Langevin algorithm (MALA). The informed proposal distribution of this new sampler features two tuning parameters: the usual step size parameter \(\sigma ^2\) and an interpolation parameter \(\gamma \) that may be adjusted to accommodate the dimension of the target distribution. We theoretically study the efficiency of the sampler by making use of the local- and global-balance concepts introduced in Zanella (JASA 115:852–865, 2020) and provide efficient tuning guidelines that work well with a variety of target distributions. Although the usual MALA (\(\gamma =1\)) is shown to be optimal for infinite-dimensional targets, in practice, the generalized MALA (\(1<\gamma \le 2\)) remains the most appealing option, even in high-dimensional contexts. Simulation studies and numerical experiments are presented to illustrate our findings. We apply the new sampler to a Bayesian logistic regression context and show that its efficiency compares favourably to competing algorithms.
Similar content being viewed by others
Data Availability Statement
Data sets used in §4 are publicly available at http://archive.ics.uci.edu (German Credit, Australian Credit, Heart) and in the package MASS on R (Pima Indian).
References
Boisvert-Beaudry, G.: Efficacité des distributions instrumentales en équilibre dans un algorithme de type Metropolis-Hastings. Université de Montréal, Thesis (2019)
Geyer, C.J.: Practical Markov chain Monte Carlo. Stat. Sci. 7(4), 473–483 (1992)
Girolami, M., Calderhead, B.: Riemann manifold Langevin and Hamiltonian Monte Ccarlo methods. J. R. Stat. Soc. Ser. B 73(2), 123–214 (2011)
Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)
Michie, D., Spiegelhalter, D.J., Taylor, C.C., Campbell, J. (eds.): Machine learning, neural and statistical classification. Ellis Horwood, Upper Saddle River, NJ, USA (1994)
Peskun, P.H.: Optimum Monte-Carlo sampling using Markov chains. Biometrika 60(3), 607–612 (1973)
Ripley, B.D., Hjort, N.: Pattern recognition and neural networks. Cambridge University Press, Cambridge (1996)
Roberts, G., Rosenthal, J.: Optimal scaling for various Metropolis-Hastings algorithms. Stat. Sci. 16(4), 351–367 (2001)
Roberts, G.O., Gelman, A., Gilks, W.R.: Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probabil. 7, 110–120 (1997)
Roberts, G.O., Rosenthal, J.S.: Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 60(1), 255–268 (1998)
Tierney, L.: A note on Metropolis-Hastings kernels for general state spaces. Annals Appl. Probabil. , 1–9 (1998)
Zanella, G.: Informed proposals for local MCMC in discrete spaces. J. Am. Stat. Assoc. 115(530), 852–865 (2020)
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors are grateful to the two anonymous reviewers whose suggestions and comments led to an improved manuscript. They also acknowledge the support of the Natural Sciences and Engineering Research Council of Canada
Rights and permissions
About this article
Cite this article
Boisvert-Beaudry, G., Bédard, M. MALA with annealed proposals: a generalization of locally and globally balanced proposal distributions. Stat Comput 32, 5 (2022). https://doi.org/10.1007/s11222-021-10063-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11222-021-10063-1
Keywords
- Bayesian logistic regression
- Informed proposal distribution
- Markov chain Monte Carlo
- Metropolis-adjusted Langevin algorithm
- Reversibility