Abstract
Document Classification is essential to information retrieval and text mining. Accuracy and interpretability are two important aspects of text classifiers. This paper proposes an interpretable classification method (DLDPvMFs) by using the Dirichlet process mixture (DPM) model to discover the hidden topics distinctly within each label for classification of directional data based on the von Mises-Fisher (vMF) distribution, which arises naturally for data distributed on the unit hypersphere. We use a mean-field variational inference algorithm when developing DLDPvMFs. By using the label information of the training data explicitly and determining automatically the number of topics for each label to find the topical space, class topics are coherent, relevant and discriminative and since they help us interpret class’s label as well as distinguish classes. Our experimental results showed the advantages of our approach via significant criteria such as separability, interpretability and effectiveness in classification task of large datasets with high dimension and complex distribution. Our obtained results are highly competitive with state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Van de Merckt, T., Decaestecker, C.: About breaking the trade off between accuracy and comprehensibility in concept learning. In: IJCAI 1995 Workshop on Machine Learning and Comprehensibility (1995)
Anh, N.K., Tam, N.T., Linh, N.V.: Document clustering using dirichlet process mixture model of von mises-fisher distributions. In: 4th International Symposium on Information and Communication Technology, SoICT 2013, pp. 131–138 (2013)
Anh, N.K., Van Linh, N., Ky, L.H., et al.: Document classification using semi-supervived mixture model of von mises-fisher distributions on document manifold. In: Proceedings of the Fourth Symposium on Information and Communication Technology, pp. 94–100. ACM (2013)
Anh, N.K., Linh, N.V., Tam, N.T.: Document clustering using mixture model of von mises-fisher distributions on document manifold, pp. 146–151 (2013)
Gopal, S., Yang, Y.: Von mises-fisher clustering models. In: Proceedings of The 31st International Conference on Machine Learning, pp. 154–162 (2014)
Ramage, D., Hall, D., Nallapati, R., Manning, C.D.: Labeled lda: A supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 248–256. Association for Computational Linguistics (2009)
Ahmed, A., Xing, E.P.: Staying informed: supervised and semi-supervised multi-view topical analysis of ideological perspective. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1140–1150. Association for Computational Linguistics (2010)
Ramage, D., Manning, C.D., Dumais, S.: Partially labeled topic models for interpretable text mining. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 457–465. ACM (2011)
Banerjee, A., Dhillon, I.S., Ghosh, J., Sra, S.: Clustering on the unit hypersphere using von mises-fisher distributions. J. Mach. Learn. Res. 6, 1345–1382 (2005)
Zhu, J., Ahmed, A., Xing, E.P.: Medlda: maximum margin supervised topic models. The Journal of Machine Learning Research 13(1), 2237–2278 (2012)
Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. The Annals of Statistics 1(2), 209–230 (1973)
Sethuraman, J.: A constructive definition of Dirichlet priors. Statistica Sinica 4, 639–650 (1994)
Mardia, K.V., Atoum, E.S.A.M.: Bayesian inference for the von Mises-Fisher distribution. Biometrika 63, 203–206 (1976)
Ishwaran, H., James, L.F.: Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association 96(453), 161–173 (2001)
Neal, R.M.: Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics 9(2), 249–265 (2000)
Blei, D.M., Jordan, M.I.: Variational inference for dirichlet process mixtures. Bayesian Analysis 1(1), 121–144 (2006)
Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of Machine Learning Research 9(11) (2008)
Niyogi, P.: Manifold regularization and semi-supervised learning: some theoretical analyses. Journal of Machine Learning Research 14(1), 1229–1250 (2013)
Than, K., Ho, T.B., Nguyen, D.K.: An effective framework for supervised dimension reduction. Neurocomputing 139, 397–407 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Van Linh, N., Anh, N.K., Than, K., Tat, N.N. (2015). Effective and Interpretable Document Classification Using Distinctly Labeled Dirichlet Process Mixture Models of von Mises-Fisher Distributions. In: Renz, M., Shahabi, C., Zhou, X., Cheema, M. (eds) Database Systems for Advanced Applications. DASFAA 2015. Lecture Notes in Computer Science(), vol 9050. Springer, Cham. https://doi.org/10.1007/978-3-319-18123-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-18123-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18122-6
Online ISBN: 978-3-319-18123-3
eBook Packages: Computer ScienceComputer Science (R0)