Abstract
Detecting local topics from social media is an important task for many applications, ranging from event tracking to emergency warning. Recent years have witnessed growing interest in leveraging multi-modal social media information for local topic detection. However, existing methods suffer great limitation in capturing comprehensive semantics from social media and fall short in bridging semantic gaps among multi-modal contents, i.e., some of them overlook visual information which contains rich semantics, others neglect indirect semantic correlation among multi-modal information. To deal with above problems, we propose an effective local topic detection method with two major modules, called IEMM-LTD. The first module is an image-enhanced multi-modal embedding learner to generate embeddings for words and images, which can capture comprehensive semantics and preserve both direct and indirect semantic correlations. The second module is an embedding based topic model to detect local topics represented by both words and images, which adopts different prior distributions to model multi-modal information separately and can find the number of topics automatically. We evaluate the effectiveness of IEMM-LTD on two real-world tweet datasets, the experimental results show that IEMM-LTD has achieved the best performance compared to the existing state-of-the-art methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anderson, P., et al.: Bottom-up and top-down attention for image captioning and visual question answering. In: CVPR, pp. 6077–6086. IEEE Computer Society (2018)
Bao, J., Duan, N., Zhou, M.: Knowledge-based question answering as machine translation. In: ACL (1). The Association for Computer Linguistics (2014)
Batmanghelich, K.N., Saeedi, A., Narasimhan, K., Gershman, S.: Nonparametric spherical topic modeling with word embeddings. In: ACL (2). The Association for Computer Linguistics (2016)
Chen, F., Xie, S., Li, X., Li, S.: What topics do images say: A neural image captioning model with topic representation. In: ICME Workshops, IEEE (2019)
Chen, J., Gao, N., Xue, C., Tu, C., Zha, D.: Perceiving topic bubbles: local topic detection in spatio-temporal tweet stream. In: Li, G., Yang, J., Gama, J., Natwichai, J., Tong, Y. (eds.) DASFAA 2019. LNCS, vol. 11447, pp. 730–747. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-18579-4_43
Chen, J., Gao, N., Xue, C., Zhang, Y.: The application of network based embedding in local topic detection from social media. In: ICTAI, pp. 1311–1319. IEEE (2019)
Cheng, Z., Bai, F., Xu, Y., Zheng, G., Pu, S., Zhou, S.: Focusing attention: Towards accurate text recognition in natural images. In: ICCV, pp. 5086–5094. IEEE Computer Society (2017)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (2019)
Duan, J., Ai, Y., Li, X.: LDA topic model for microblog recommendation. In: IALP, pp. 185–188. IEEE (2015)
Fang, A., Macdonald, C., Ounis, I., Habel, P.: Topics in tweets: a user study of topic coherence metrics for twitter data. In: Ferro, N. (ed.) ECIR 2016. LNCS, vol. 9626, pp. 492–504. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1_36
Giannakopoulos, K., Chen, L.: Incremental and adaptive topic detection over social media. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds.) DASFAA 2018. LNCS, vol. 10827, pp. 460–473. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91452-7_30
Huang, S., Chen, H., Dai, X., Chen, J.: Non-linear learning for statistical machine translation. In: ACL (1). The Association for Computer Linguistics (2015)
Jégou, S., Drozdzal, M., Vázquez, D., Romero, A., Bengio, Y.: The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In: CVPR Workshops, pp. 1175–1183 (2017)
Li, G., Zhu, L., Liu, P., Yang, Y.: Entangled transformer for image captioning. In: ICCV, pp. 8927–8936. IEEE (2019)
Liu, H., Ge, Y., Zheng, Q., Lin, R., Li, H.: Detecting global and local topics via mining twitter data. Neurocomputing 273 120-132 (2018)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015)
Morstatter, F., Liu, H.: In search of coherence and consensus: measuring the interpretability of statistical topics. J. Mach. Learn. Res. 18(169), 1–32 (2017)
Muhammad, U., Wang, W., Chattha, S.P., Ali, S.: Pre-trained vggnet architecture for remote-sensing image scene classification. In: ICPR, pp. 1622–1627. IEEE Computer Society (2018)
Qian, S., Zhang, T., Xu, C., Shao, J.: Multi-modal event topic model for social event analysis. IEEE Trans. Multimedia 18(2), 233–246 (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
Singh, V.K., et al.: Classification of breast cancer molecular subtypes from their micro-texture in mammograms using a vggnet-based convolutional neural network. In: CCIA, vol. 300, pp. 76–85. IOS Press (2017)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Wold, H.M., Vikre, L., Gulla, J.A., Özgöbek, Ö., Su, X.: Twitter topic modeling for breaking news detection. In: WEBIST (2), pp. 211–218. SciTePress (2016)
Yan, X., Guo, J., Lan, Y., Cheng, X.: A biterm topic model for short texts. In: WWW, pp. 1445–1456. International World Wide Web Conferences Steering Committee/ACM (2013)
Zhang, C., et al.: Regions, periods, activities: Uncovering urban dynamics via cross-modal representation learning. In: WWW, pp. 361–370. ACM (2017)
Zhang, C., Lu, S., Zhang, C., Xiao, X., Wang, Q., Chen, G.: A novel hot topic detection framework with integration of image and short text information from twitter. IEEE Access 7, 9225–9231 (2019)
Zhao, W.X., et al.: Comparing Twitter and traditional media using topic models. In: Clough, P. (ed.) ECIR 2011. LNCS, vol. 6611, pp. 338–349. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20161-5_34
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J., Gao, N., Zhang, Y., Tu, C. (2021). Image-Enhanced Multi-Modal Representation for Local Topic Detection from Social Media. In: Jensen, C.S., et al. Database Systems for Advanced Applications. DASFAA 2021. Lecture Notes in Computer Science(), vol 12682. Springer, Cham. https://doi.org/10.1007/978-3-030-73197-7_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-73197-7_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73196-0
Online ISBN: 978-3-030-73197-7
eBook Packages: Computer ScienceComputer Science (R0)