Abstract
Answer natural language questions on knowledge bases (\(\mathsf {KBQA}\)) has attracted wide attention. Several techniques have been developed for answering simple questions. These techniques mostly rely on deep networks to perform classification for relation prediction. Nowadays, contrastive learning has shown its powers in improving performances of classification, while most prior techniques do not gain benefit from this. In light of these, we propose a novel approach to answering simple questions on knowledge bases. Our approach has two key features. (1) It leverages pre-trained transformers to gain better performance on entity linking. (2) It employs a contrastive learning based model for relation prediction. We experimentally verify the performance of our approach, and show that our approach achieves an accuracy of 83.54%, which beats existing state-of-the-art techniques, on a typical benchmark dataset; we also conduct a deep analysis to show advantages of our technique, especially its sub-modules.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Removed Questions. https://github.com/Mycatinjuly/SimpleQuestion_QA
Bast, H., Haussmann, E.: More accurate question answering on freebase. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 1431–1440 (2015)
Berant, J., Chou, A., Frostig, R., Liang, P.: Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1533–1544. ACL (2013)
Bollacker, K.D., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)
Bordes, A., Chopra, S., Weston, J.: Question answering with subgraph embeddings. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 615–620 (2014)
Bordes, A., Usunier, N., Chopra, S., Weston, J.: Large-scale simple question answering with memory networks. CoRR abs/1506.02075 (2015)
Bordes, A., Weston, J., Usunier, N.: Open question answering with weakly supervised embedding models. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8724, pp. 165–180. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44848-9_11
Cai, Q., Yates, A.: Large-scale semantic parsing via schema matching and lexicon extension. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 423–433 (2013)
Carlsson, F., Gyllensten, A.C., Gogoulou, E., Hellqvist, E.Y., Sahlgren, M.: Semantic re-tuning with contrastive tension. In: International Conference on Learning Representations (2020)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)
Cheng, L., Chen, Z., Ren, J.: Enhancing question answering over knowledge base using dynamical relation reasoning. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2020)
Deng, Y., Zhang, W., Lam, W.: Learning to rank question answer pairs with bilateral contrastive data augmentation. In: Proceedings of the Seventh Workshop on Noisy User-generated Text, pp. 175–181 (2021)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)
Dong, L., Wei, F., Zhou, M., Xu, K.: Question answering over freebase with multi-column convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 260–269 (2015)
Fader, A., Zettlemoyer, L.S., Etzioni, O.: Paraphrase-driven learning for open question answering. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pp. 1608–1618 (2013)
Gupta, V., Chinnakotla, M., Shrivastava, M.: Retrieve and re-rank: a simple and effective IR approach to simple question answering over knowledge graphs. In: Proceedings of the First Workshop on Fact Extraction and VERification), pp. 22–27, November 2018
Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 1735–1742. IEEE (2006)
Hao, Y., Liu, H., He, S., Liu, K., Zhao, J.: Pattern-revising enhanced simple question answering over knowledge bases. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 3272–3282 (2018)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 328–339 (2018)
Lehmann, J., et al.: DBpedia - a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web 6(2), 167–195 (2015)
Lin, T., Goyal, P., Girshick, R.B., He, K., Dollár, P.: Focal loss for dense object detection. In: IEEE International Conference on Computer Vision, pp. 2999–3007 (2017)
Lukovnikov, D., Fischer, A., Lehmann, J.: Pretrained transformers for simple question answering over knowledge graphs. In: Ghidini, C., et al. (eds.) ISWC 2019. LNCS, vol. 11778, pp. 470–486. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30793-6_27
Lukovnikov, D., Fischer, A., Lehmann, J., Auer, S.: Neural network-based question answering over knowledge graphs on word and character level. In: Barrett, R., Cummings, R., Agichtein, E., Gabrilovich, E. (eds.) Proceedings of the 26th International Conference on World Wide Web, pp. 1211–1220. ACM (2017)
Luo, D., Su, J., Yu, S.: A Bert-based approach with relation-aware attention for knowledge base question answering. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2020)
Maheshwari, G., Trivedi, P., Lukovnikov, D., Chakraborty, N., Fischer, A., Lehmann, J.: Learning to rank query graphs for complex question answering over knowledge graphs. In: Ghidini, C., et al. (eds.) ISWC 2019. LNCS, vol. 11778, pp. 487–504. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30793-6_28
Mohammed, S., Shi, P., Lin, J.: Strong baselines for simple question answering over knowledge graphs with and without neural networks. In: Walker, M.A., Ji, H., Stent, A. (eds.) Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 291–296. Association for Computational Linguistics (2018)
Qin, Y., et al.: ERICA: improving entity and relation understanding for pre-trained language models via contrastive learning. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pp. 3350–3363 (2021)
Qiu, C., Zhou, G., Cai, Z., Søgaard, A.: A global-local attentive relation detection model for knowledge-based question answering. IEEE Trans. Artif. Intell. 2(2), 200–212 (2021)
Qu, Y., Liu, J., Kang, L., Shi, Q., Ye, D.: Question answering over freebase via attentive RNN with similarity matrix based CNN. CoRR abs/1804.03317 (2018)
Rao, J., He, H., Lin, J.: Noise-contrastive estimation for answer selection with deep neural networks. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 1913–1916 (2016)
Reddy, S., Lapata, M., Steedman, M.: Large-scale semantic parsing without question-answer pairs. Trans. Assoc. Comput. Linguist. 2, 377–392 (2014)
Sharath, J.S., Rekabdar, B.: Question answering over knowledge base using language model embeddings. In: 2020 International Joint Conference on Neural Networks, pp. 1–8. IEEE (2020)
Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web, pp. 697–706 (2007)
Sun, Y., et al.: Semantic parsing with syntax- and table-aware SQL generation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 361–372 (2018)
Wang, Y., Zhang, R., Xu, C., Mao, Y.: The APVA-TURBO approach to question answering in knowledge base. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 1998–2009 (2018)
Yao, X., Durme, B.V.: Information extraction over structured data: question answering with freebase. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 956–966 (2014)
Yih, W., Chang, M., He, X., Gao, J.: Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics, pp. 1321–1331 (2015)
Yin, W., Yu, M., Xiang, B., Zhou, B., Schütze, H.: Simple question answering by attentive convolutional neural network. In: COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, pp. 1746–1756 (2016)
Yu, M., Yin, W., Hasan, K.S., dos Santos, C.N., Xiang, B., Zhou, B.: Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 571–581 (2017)
Zhang, Y., He, R., Liu, Z., Lim, K.H., Bing, L.: An unsupervised sentence embedding method by mutual information maximization. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1601–1610 (2020)
Zhang, Y., Jin, L., Zhang, Z., Li, X., Liu, Q., Wang, H.: SF-ANN: leveraging structural features with an attention neural network for candidate fact ranking. Appl. Intell. 1–16 (2021)
Acknowledgement
This work is supported by Sichuan Scientific Innovation Fund (No. 2022JDRC0009) and the National Key Research and Development Program of China (No. 2017YFA0700800).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, X., Yang, L., He, H., Fang, Y., Zhan, H., Zhang, J. (2022). Enhanced Simple Question Answering with Contrastive Learning. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13368. Springer, Cham. https://doi.org/10.1007/978-3-031-10983-6_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-10983-6_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10982-9
Online ISBN: 978-3-031-10983-6
eBook Packages: Computer ScienceComputer Science (R0)