Abstract
Chinese named entity recognition has been widely used in many fields, such as species recognition in marine information, and so on. Compared with the standard named entity recognition (NER), the performance of the Chinese marine named entity recognition is low, which is mainly limited by the normative nature of the text and the scale of the tagged corpus. In recent years, the research of named entity recognition primarily focuses on the small scale of the tagged corpus. It tends to use external knowledge or use joint training to improve final recognition performance. However, there is little research on the problem that the accuracy of NER will be suboptimal if the model is trained inadequately. To this end, in order to improve the accuracy of naming entity recognition and recognize more entities in corpus, this paper proposes a named entity recognition method that combines knowledge graph embedding with a self-attention mechanism. The entity embeddings of the marine knowledge graph (KG) are empolyed for the hidden units of NER model with attention mechanism in an end-to-end way. Therefore, the model can get additional auxiliary information to improve performance. Lastly, we conduct extensive experiments on marine corpus and other public datasets. The experimental results verify the effectiveness of our proposed method.
Similar content being viewed by others
References
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795
Bunescu R C, Mooney R J (2005) A shortest path dependency kernel for relation extraction. In: EMNLP, pp 724–731
Cao P, Chen Y, Liu K, Zhao J, Liu S (2018) Adversarial transfer learning for chinese named entity recognition with self-attention mechanism. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 182–192
Chen L, Yang M (2017) Semi-supervised dictionary learning with label propagation for image classification. Comput Vis Media 3(1):83–94
Chen Y, Xu L, Liu K, Zeng D, Zhao J (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: ACL, pp 167–176
Chorowski J K, Bahdanau D, Serdyuk D, Cho K, Bengio Y (2015) Attention-based models for speech recognition. In: Advances in neural information processing systems, pp 577–585
Devlin J, Chang M -W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805
Ebisu T, Ichise R (2018) Toruse: knowledge graph embedding on a lie group. In: Thirty-second AAAI conference on artificial intelligence
Fader A, Zettlemoyer L, Etzioni O (2013) Paraphrase-driven learning for open question answering. In: ACL, pp 1608–1618
Fan D -P, Cheng M -M, Liu J -J, Gao S -H, Hou Q, Borji A (2018) Salient objects in clutter: bringing salient object detection to the foreground. In: Proceedings of the European conference on computer vision (ECCV), pp 186–202
Forney G D (1973) The viterbi algorithm. Proc IEEE 61(3):268–278
Fu K, Zhao Q, Gu I Y -H, Yang J (2019) Deepside: a general deep framework for salient object detection. Neurocomputing 356:69–82
Fu K, Fan D -P, Ji G -P, Zhao Q (2020) Jl-dcf: joint learning and densely-cooperative fusion framework for rgb-d salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3052–3062
Greenberg N, Bansal T, Verga P, McCallum A (2018) Marginal likelihood training of bilstm-crf for biomedical named entity recognition from disjoint label sets. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2824–2829
Guo S, Wang Q, Wang L, Wang B, Guo L (2018) Knowledge graph embedding with iterative guidance from soft rules. In: Thirty-second AAAI conference on artificial intelligence
Han S, Hao X, Huang H (2018) An event-extraction approach for business analysis from online chinese news. Electron Commer Res Appl 28:244–260
He H, Sun X (2016) F-score driven max margin neural network for named entity recognition in chinese social media. arXiv:1611.04234
He H, Sun X (2017) A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In: Thirty-first AAAI conference on artificial intelligence
Hochreiter S, Schmidhuber J (1997) Lstm can solve hard long time lag problems. In: Advances in neural information processing systems, pp 473–479
Huang Z, Xu W, Yu K (2015) Bidirectional lstm-crf models for sequence tagging. arXiv:1508.01991
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167
Ju M, Miwa M, Ananiadou S (2018) A neural layered model for nested named entity recognition. In: Proceedings of the 2018 conference of the North American Chapter Of The Association For Computational Linguistics: human language technologies, vol 1 (Long Papers), pp 1446–1459
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980
Lafferty J, McCallum A, Pereira F C (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data
Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 260–270
Lee D, Yu W, Lim H (2017) Bi-directional lstm-cnn-crf for korean named entity recognition system with feature augmentation. J Korea Converg Soc 8(12):55–62
Lei Ba J, Kiros J R, Hinton G E (2016) Layer normalization. arXiv:1607.06450
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI conference on artificial intelligence
Liu W, Xu T, Xu Q, Song J, Zu Y (2019) An encoding strategy based word-character lstm for Chinese ner. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1 (Long and Short Papers), pp 2379–2389
Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3623–3632
Lu X, Wang W, Shen J, Tai Y -W, Crandall D J, Hoi S C (2020) Learning video object segmentation from unlabeled videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8960–8970
McClosky D, Surdeanu M, Manning C D (2011) Event extraction as dependency parsing. In: HLT, pp 1626–1635
Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119
Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. In: ACL, pp 1105–1116
Ng A Y (2004) Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In: Proceedings of the twenty-first international conference on machine learning. ACM, p 78
Park G, Kim H (2018) Low-cost implementation of a named entity recognition system for voice-activated human-appliance interfaces in a smart home. Sustainability 10(2):488
Peng N, Dredze M (2015) Named entity recognition for chinese social media with jointly trained embeddings. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 548–554
Peng N, Dredze M (2016) Improving named entity recognition for chinese social media with word segmentation representation learning. arXiv:1603.00786
Peng D, Wang Y, Liu C, Chen Z (2019) Tl-ner: a transfer learning model for chinese named entity recognition. Information Systems Frontiers 1–14
Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Singh S, Riedel S, Martin B, Zheng J, McCallum A (2013) Joint inference of entities, relations, and coreference. In: AKBC, pp 1–6
Upadhyay S, Gupta N, Roth D (2018) Joint multilingual supervision for cross-lingual entity linking. In: EMNLP, pp 2486–2495
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Voorhees E, Harman D K (2006) Trec : experiment and evaluation in information retrieval. J Am Soc Inf Sci Technol 32(4):563–567
Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI conference on artificial intelligence
Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29 (12):2724–2743
Wang W, Lu X, Shen J, Crandall D J, Shao L (2019) Zero-shot video object segmentation via attentive graph neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 9236–9245
Xiang Y, et al. (2017) Chinese named entity recognition with character-word mixed embedding. In: Proceedings of the 2017 ACM on conference on information and knowledge management. ACM, pp 2055–2058
Xiong C, Power R, Callan J (2017) Explicit semantic ranking for academic search via knowledge graph embedding. In: Proceedings of the 26th international conference on world wide web, pp 1271–1279
Xin J, Lin Y, Liu Z, Sun M (2018) Improving neural fine-grained entity typing with knowledge attention. In: AAAI, pp 1–8
Xu C, Wang F, Han J, Li C (2019) Exploiting multiple embeddings for chinese named entity recognition. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 2269–2272
Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K (2020) Product knowledge graph embedding for e-commerce. In: Proceedings of the 13th international conference on web search and data mining, pp 672–680
Yao X, Van Durme B (2014) Information extraction over structured data: question answering with freebase. In: ACL, pp 956–966
Yao Y, Rosasco L, Caponnetto A (2007) On early stopping in gradient descent learning. Construct Approx 26(2):289–315
Yao L, Torabi A, Cho K, Ballas N, Pal C, Larochelle H, Courville A (2015) Video description generation incorporating spatio-temporal features and a soft-attention mechanism. arXiv:1502.08029
Yubo C, Liheng X, Kang L, Daojian Z, Jun Z et al (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: ACL, pp 167–176
Zhang Y, Yang J (2018) Chinese ner using lattice lstm. arXiv:1805.02023
Zhao J -X, Cao Y, Fan D -P, Cheng M -M, Li X -Y, Zhang L (2019) Contrast prior and fluid pyramid integration for rgbd salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3927–3936
Acknowledgments
This work is supported by the Second Level Research Project of China Geological Survey (Grant No. DD20191008).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
He, S., Sun, D. & Wang, Z. Named entity recognition for Chinese marine text with knowledge-based self-attention. Multimed Tools Appl 81, 19135–19149 (2022). https://doi.org/10.1007/s11042-020-10089-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-10089-z