Abstract
A pseudo-marginal Markov chain Monte Carlo (PMCMC) method is proposed for nonnegative matrix factorization (NMF). The sampler jointly simulates the joint posterior distribution for the nonnegative matrices and the matrix dimensions which indicate the number of the nonnegative components in the NMF model. We show that the PMCMC sampler is a generalization of a version of the reversible jump Markov chain Monte Carlo. An illustrative synthetic data was used to demonstrate the ability of the proposed PMCMC sampler in inferring the nonnegative matrices and as well as the matrix dimensions. The proposed sampler was also applied to a nuclear magnetic resonance spectroscopy data to infer the number of nonnegative components.
Similar content being viewed by others
References
Andrieu C, Roberts GO (2009) The pseudo-marginal approach for efficient Monte Carlo computations. Ann Stat 37(2):697–725
Andrieu C, Doucet A, Holenstein R (2010) Particle Markov chain Monte Carlo methods (with discussion). J R Stat Soc B 72(3):269–342
Boutsidis C, Zouzias A, Mahoney MW, Drineas P (2015) Randomized dimensionality reduction for-means clustering. IEEE Trans Inf Theory 61(2):1045–1062
Calderhead B, Girolami M (2009) Estimating Bayes factors via thermodynamic integration and population MCMC. Comput Stat Data Anal 53:4028–4045
Chib S (1995) Marginal likelihood from the Gibbs output. J Am Stat Assoc 90:1313–1321
Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 126–135
Dunn WB, Bailey N, Johnson HE (2005) Measuring the metabolome: current analytical technologies. Analyst 130:606–625
Friel N, Pettitt AN (2008) Marginal likelihood estimation via power posteriors. J R Stat Soc B 70:589–607
Green P (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82:711–732
Liu JS (2001) Monte Carlo strategies in scientific computing. Springer, Berlin
Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461
Liu T, Tao D (2015) On the performance of manhattan nonnegative matrix factorization. IEEE Trans Neural Netw Learn Syst 99:1
Manganas L, Zhang X, Li Y, Hazel RD, Smith SD, Wagshul ME, Henn F, Benveniste H, Djuric PM, Enikolopov G, Maletic-Savatic M (2007) Magnetic resonance spectroscopy identifies neural progenitor cells in the live human brain. Science 318:980–985
Rubin D (1987) Comment on the calculation of posterior distributions by data augmentation by Tanner, M.A. and Wong, W. H. J Am Stat Assoc 82:543
Schachtner R, Po G, Tomé AM, Puntonet CG, Lang EW et al (2014) A new bayesian approach to nonnegative matrix factorization: uniqueness and model order selection. Neurocomputing 138:142–156
Schmidt MN, Winther O, Hansen LK (2009) Bayesian non-negative matrix factorization. In: International conference on independent component analysis and signal separation. Lecture Notes in Computer Science (LNCS), vol 5441. Springer, Berlin, pp 540–547
Sun M, Zhang X et al (2015) A stable approach for model order selection in nonnegative matrix factorization. Pattern Recognit Lett 54:97–102
Tan V, Fevotte C (2012) Automatic relevance determination in nonnegative matrix factorization with the beta-divergence. IEEE Trans Pattern Anal Mach Intell 35:1592–1605
Xu C, Tao D, Xu C, Rui Y (2014) Large-margin weakly supervised dimensionality reduction. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 865–873
Zhong M, Girolami M (2009) Reversible jump MCMC for non-negative matrix factorization. In: Proceedings of the 12th international conference on artificial intelligence and statistics (AISTATS), FL, pp 663–670
Zhong M, Girolami M, Faulds K, Graham D (2011) Bayesian methods to detect dye-labelled DNA oligonucleotides in multiplexed Raman spectra. J R Stat Soc C (Appl Stat) 60(2):187–206
Acknowledgments
This work is supported by the fundamental research funds for the central universities in China.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Du, J., Zhong, M. Pseudo-marginal Markov Chain Monte Carlo for Nonnegative Matrix Factorization. Neural Process Lett 45, 553–562 (2017). https://doi.org/10.1007/s11063-016-9542-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-016-9542-x