Abstract
In recent years, with the development of deep learning and the increasing demand for medical information acquisition in medical information technology applications such as clinical decision support, Clinical Event Detection has been widely studied as its subtask. However, directly applying advances in deep learning to Clinical Event Detection tasks often produces undesirable results. This paper proposes a multi-granularity information fusion encoder-decoder framework that introduces external knowledge. First, the word embedding generated by the pre-trained biomedical language representation model (BioBERT) and the character embedding generated by the Convolutional Neural Network are spliced. And then perform Part-of-Speech attention coding for character-level embedding, perform semantic Graph Convolutional Network coding for the spliced character-word embedding. Finally, the information of these three parts is fused as Conditional Random Field input to generate the sequence label of the word. The experimental results on the 2012 i2b2 data set show that the model in this paper is superior to other existing models. In addition, the model in this paper alleviates the problem that “occurrence” event type seem more difficult to detect than other event types.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
Yan, H., Deng, B., Li, X., et al.: TENER: adapting transformer encoder for named entity recognition. arXiv preprint arXiv:1911.04474 (2019)
Mikolov, T., Chen, K., Corrado, G., et al.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543 (2014)
Lee, J., Yoon, W., Kim, S., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)
Lample, G., Ballesteros, M., Subramanian, S., et al.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270. Association for Computational Linguistics, San Diego (2016)
Ma, X., Hovy, E.: End-to-End sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1064–1074. Association for Computational Linguistics, Berlin (2016)
Liu, L., Shang, J., Ren, X., et al.: Empower sequence labeling with task-aware neural language model. Proc. AAAI Conf. Artif. Intell. 32(1), 5253–5260 (2018)
Chiu, J., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Ling. 4, 357–370 (2016)
Grouin, C., Grabar, N., Hamon, T., et al.: Eventual situations for timeline extraction from clinical reports. J. Am. Med. Inform. Assoc. 20(5), 820–827 (2013)
Fries, J.: Brundlefly at SemEval-2016 Task 12: recurrent neural networks vs. joint inference for clinical temporal information extraction. In: Proceedings of the 10th International Workshop on Semantic Evaluation(SemEval-2016), pp. 1274–1279. Association for Computational Linguistics, San Diego (2016)
Cheng, F., Miyao, Y.: Classifying temporal relations by bidirectional LSTM over dependency paths. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 1–6. Association for Computational Linguistics, Vancouver (2017)
Dligach, D., Miller, T., Lin, C., et al.: Neural temporal relation extraction. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 746–751. Association for Computational Linguistics, Valencia (2017)
Li, P., Huang, H.: UTA DLNLP at SemEval-2016 Task 12: deep learning based natural language processing system for clinical information identification from clinical notes and pathology reports. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1268–1273. Association for Computational Linguistics, San Diego (2016)
Tourille, J., Ferret, O., Neveol, A., et al.: Neural architecture for temporal relation extraction: a BI-LSTM approach for detecting narrative containers. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 224–230 (2017)
Lin, C., Miller, T., Dligach, D., et al.: Self-training improves recurrent neural networks performance for temporal relation extraction. In: Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, pp. 165–176. Association for Computational Linguistics, Brussels (2016)
Zhao, S., Li, L., Lu, H., et al.: Associative attention networks for temporal relation extraction from electronic health records. J. Biomed. Inform. 99(103309) (2019)
Cortes, C., Vapnik, V.: Support-vector networks. In: Proceedings of the Twelfth International Conference on Machine Learning, vol. 20, no. 3, pp. 273–297 (1995)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282–289 (2001)
Roberts, K., Rink, B., Harabagiu, S.: A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text. J. Am. Med. Inform. Assoc. 20(5), 867–875 (2013)
Kovacevic, A., Dehghan, A., Filannino, M., et al.: Combining rules and machine learning for extraction of temporal expressions and events from clinical narratives. J. Am. Med. Inform. Assoc. 20(5), 820–827 (2013)
Zhu, H., Paschalidis, I., Tahmasebi, A.: Clinical concept extraction with contextual word embedding. J. Am. Med. Inform. Assoc. 26(11), 1297–1304 (2019)
Akhundov, A., Trautmann, D., Groh, G.: Sequence labeling: a practical approach. arXiv preprint arXiv:1808.03926 (2018)
Chen, H., Lin, Z., Ding, G., et al.: GRN: gated relation network to enhance convolutional neural network for named entity recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (2019). https://doi.org/10.1609/aaai.v33i01.33016236
Lin, Y., Lee, D., Shen, M., et al.: TriggerNER: learning with entity triggers as explanations for named entity recognition. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8503–8511. Association for Computational Linguistics, Online (2020)
Liu, P., Chang, S., Huang, X., et al.: Contextualized non-local neural networks for sequence learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 1, pp. 6762–6769 (2019)
Lin, Z., Feng, M., Santos, C.: A structured self-attentive sentence embedding. In: International Conference on Learning Representations (ICLR) (2017)
Luo, Y., Zhao, H.: Bipartite flat-graph network for nested named entity recognition. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6408–6418. Association for Computational Linguistics, Online (2020)
Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)
Qian, Y., Santus, E., Jin, Z., et al.: Graphie: a graph-based framework for information extraction. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 751–761. Association for Computational Linguistics, Minneapolis (2019)
Marcheggiani, D., Titov, I.: Encoding sentences with graph convolutional networks for semantic role labeling. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1506–1515. Association for Computational Linguistics, Copenhagen (2017)
Fu, T., Li, P., Filannino, M., Ma, W.: GraphRel: modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1409–1418. Association for Computational Linguistics, Florence (2019)
Smith, L.: Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 464–472. IEEE, Santa Rosa (2017)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, D., Zhang, Z., Peng, H., Han, R. (2021). GCN with External Knowledge for Clinical Event Detection. In: Li, S., et al. Chinese Computational Linguistics. CCL 2021. Lecture Notes in Computer Science(), vol 12869. Springer, Cham. https://doi.org/10.1007/978-3-030-84186-7_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-84186-7_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84185-0
Online ISBN: 978-3-030-84186-7
eBook Packages: Computer ScienceComputer Science (R0)