Abstract
Constituency parsing is the process of analyzing a sentence by breaking it down into sub-phrases also known as constituents. Although many deep neural models have achieved state-of-the-art results on this task, few consider entity-violating issue, i.e. an entity cannot form a complete sub-tree in the resultant constituent parsing tree. To attack this issue, this paper proposes an entity-aware biaffine attention model for constituent parsing. It leverages entity information for a potential phrase when conducting biaffine attention between the start and end words of the phrase. In the absence of the proper metric for comparison, the entity violating rate (EVR) as a new metric is introduced here to evaluate how many the final parsing trees suffer from entity violating issue. The lower the EVR, the better the model. This metric from a brand perspective helps us understand the potential of existing arts. Experiments on three publicly popular datasets including ONTONOTES, PTB and CTB show that our model achieves the lowest EVR while almost achieving the same performance in terms of the three conventional metrics, i.e., precision, recall, and F1-score. Moreover, extensive experiments of sentence sentiment analysis as a downstream application further exhibit the efficacy of our model and the validity of the proposed metric EVR.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bowman, S.R., Gauthier, J., Rastogi, A., Gupta, R., Manning, C.D., Potts, C.: A fast unified model for parsing and sentence understanding. In: Proceedings of the ACL Conference (2016)
Chen, Q., Zhu, X., Ling, Z., Wei, S., Jiang, H., Inkpen, D.: Enhanced LSTM for natural language inference. In: Proceedings of the ACL Conference (2017)
Cheng, J., Lopez, A., Lapata, M.: A generative parser with a discriminative recognition algorithm. In: Proceedings of the ACL Conference (2017)
Collins, M.: Three generative, lexicalised models for statistical parsing. In: Proceedings of the ACL Conference (1997)
Finkel, J.R., Manning, C.D.: Joint parsing and named entity recognition. In: Proceedings of the ACL Conference, pp. 326–334 (2009)
Finkel, J.R., Manning, C.D.: Hierarchical joint learning: Improving joint parsing and named entity recognition with non-jointly labeled data. In: Proceedings of the ACL Conference, pp. 720–728 (2010)
Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., Weischedel, R.: Ontonotes: the 90% solution. In: HLT-NAACL, pp. 57–60 (2006)
Jiang, M., Diesner, J.: A constituency parsing tree based method for relation extraction from abstracts of scholarly publications. In: Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing. pp. 186–191 (2019)
Kim, T., Choi, J., Edmiston, D., Bae, S., Lee, S.g.: Dynamic compositionality in recursive neural networks with structure-aware tag representations. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6594–6601 (2019)
Kitaev, N., Cao, S., Klein, D.: Multilingual constituency parsing with self-attention and pre-training. In: Proceedings of the ACL Conference, pp. 3499–3505 (2019)
Kitaev, N., Klein, D.: Constituency parsing with a self-attentive encoder. In: Proceedings of the ACL Conference (2018)
Li, D., Zhang, X., Wu, X.: Improved Chinese parsing using named entity cue. In: Proceedings of the 13th International Conference on Parsing Technologies, pp. 45–53 (2013)
Li, D., Zhang, X., Wu, X.: Integrated Chinese segmentation, parsing and named entity recognition. Chin. J. Electr. 27(4), 756–760 (2018)
Ma, C., Tamura, A., Utiyama, M., Zhao, T., Sumita, E.: Forest-based neural machine translation. In: Proceedings of the ACL Conference, pp. 1253–1263 (2018)
Mrini, K., Dernoncourt, F., Tran, Q., Bui, T., Chang, W., Nakashole, N.: Rethinking self-attention: towards interpretability in neural parsing. In: Proceedings of the EMNLP Conference, pp. 731–742 (2020)
Strzyz, M., Vilares, D., Gómez-Rodríguez, C.: Sequence labeling parsing by learning across representations. In: Proceedings of the ACL Conference, pp. 5350–5357 (2019)
Tian, Y., Song, Y., Xia, F., Zhang, T.: Improving constituency parsing with span attention. In: Proceedings of the EMNLP Conference (2020)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 6000–6010 (2017)
Vinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., Hinton, G.: Grammar as a foreign language. In: Advances in Neural Information Processing Systems, pp. 2773–2781 (2014)
Wang, R., Xin, X., Chang, W., Ming, K., Li, B., Fan, X.: Chinese NER with height-limited constituent parsing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7160–7167 (2019)
Yang, K., Deng, J.: Strongly incremental constituency parsing with graph neural networks. In: Neural Information Processing Systems (2020)
Yu, J., Bohnet, B., Poesio, M.: Named entity recognition as dependency parsing. In: Proceedings of the ACL Conference (2020)
Zhang, Y., Zhou, H., Li, Z.: Fast and accurate neural CRF constituency parsing. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI 2020), pp. 4046–4053 (2020)
Zhou, J., Zhao, H.: Head-driven phrase structure grammar parsing on PENN treebank. In: Proceedings of the ACL Conference (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bai, X., Yin, N., Zhang, X., Wang, X., Luo, Z. (2021). Entity-Aware Biaffine Attention for Constituent Parsing. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12891. Springer, Cham. https://doi.org/10.1007/978-3-030-86362-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-86362-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86361-6
Online ISBN: 978-3-030-86362-3
eBook Packages: Computer ScienceComputer Science (R0)