Abstract
The hierarchical Dirichlet processes (HDP) topic model is a Bayesian nonparametric model that provides a flexible mixed-membership to documents through topic allocation to each word. In this paper, we consider dynamic HDP topic models, in which the generative model changes in time, and develop a novel algorithm to update the posterior distribution dynamically by combining the variational inference algorithm and the power prior approach. An important advantage of the proposed algorithm is that it updates the posterior distribution by reusing a given batch algorithm without specifying a complicated dynamic generative model. Thus the proposed algorithm is conceptually and computationally simpler. By analyzing real datasets, we show that the proposed algorithm is a useful alternative approach to dynamic HDP topic identification.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We retrieved the NIPS dataset from http://www.cs.nyu.edu/~roweis/data.html in October 2020.
References
Ahmed, A., & Xing, E.P., (2010). Timeline: A dynamic hierarchical Dirichlet process model for recovering birth/death and evolution of topics in text stream. In: Proceedings of the 26th conference on uncertainty in airtificial intelligence (pp. 20–29).
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.
Fu, X., Li, J., Yang, K., Cui, L., & Yang, L. (2016). Dynamic online HDP model for discovering evolutionary topics from Chinese social texts. Neurocomputing, 171, 412–424.
Ibrahim, J. G., & Chen, M. H. (2000). Power prior distributions for regression models. Statistical Science, 15(1), 46–60.
Ibrahim, J. G., Chen, M. H., Gwon, Y., & Chen, F. (2015). The power prior: Theory and applications. Statistics in Medicine, 34(28), 3724–3749.
Ren, L., Dunson, D.B., & Carin, L., (2008). The dynamic hierarchical Dirichlet process. In: Proceedings of the 25th international conference on machine learning (pp. 824–831).
Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4, 639–650.
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006). Hierarchical Dirichlet Processes. Journal of the American Statistical Association, 101(476), 1566–1581.
Wang, C., Paisley, J.W., & Blei, D.M., (2011). Online variational inference for the hierarchical Dirichlet process. In: Proceedings of the 14th international conference on artificial intelligence and statistics (pp. 752–760).
Wang, P., Zhang, P., Zhou, C., Li, Z., & Yang, H. (2017). Hierarchical evolving Dirichlet processes for modeling nonlinear evolutionary traces in temporal data. Data Mining and Knowledge Discovery, 31(1), 32–64.
Zhang, J., Song, Y., Zhang, C., & Liu, S., (2010). Evolutionary hierarchical Dirichlet processes for multiple correlated time-varying corpora. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1079–1088).
Acknowledgements
This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. 2020R1A2C3A01003550).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jeong, K., Kim, Y. Dynamic hierarchical Dirichlet processes topic model using the power prior approach. J. Korean Stat. Soc. 50, 860–873 (2021). https://doi.org/10.1007/s42952-021-00129-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42952-021-00129-1