Abstract
Leveraging knowledge graph will benefit question answering tasks, as KG contains well-structured informative data. However, training knowledge graph-based simple question answering systems is known computationally expensive due to the complex predicate extraction and candidate pool generation. Moreover, the existing methods based on convolutional neural network (CNN) or recurrent neural network (RNN) overestimate the importance of predicate features thus reduce performance. To address these challenges, we propose a time-efficient and resource-effective framework. We use leaky n-gram to balance recall and candidate pool size in candidate pool generation. For predicate extraction, we propose a soft-histogram and self-attention (SHSA) module which serves the role of preserving the global information of questions via feature matrices. And this leads to reduce the RNN module as the simple feedforward network in predicate representation. We also designed a Hamming lower-bound label encoding algorithm to encode the label representations in lower dimensions. Experiments on benchmark datasets show that our method outperforms the competitive work for end-tasks and achieves better recall with a significantly pruned candidate space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Please note that we generate the i-th predicate embedding via Bernoulli distribution
Torch7 is utilized for implementation. The deep infrastructure was trained on a server with a single Titan Xp GPU, Intel i7-6800K CPU, 3.4 GHz, 6 cores, 12 processors, 64GB memory, Ubuntu 18.04.1 LTS. Virtuoso [8] was used as the RDF engine.
References
Bishop CM (2007) Pattern recognition and machine learning, 5th Edition. Information science and statistics. Springer http://www.worldcat.org/oclc/71008143
Bollacker KD, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, pp. 1247–1250 (2008).https://doi.org/10.1145/1376616.1376746
Bordes A, Usunier N, Chopra S, Weston J (2015) Large-scale simple question answering with memory networks. CoRR abs/1506.02075. arXiv: org/abs/1506.02075
Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder–decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734. Association for Computational Linguistics. https://doi.org/10.3115/v1/D14-1179. http://aclweb.org/anthology/D14-1179
Collobert R, Weston J (2008) A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, ICML ’08, pp. 160–167. ACM, New York, NY, USA.https://doi.org/10.1145/1390156.1390177. http://doi.acm.org/10.1145/1390156.1390177
Dai Z, Li L, Xu W (2016) Cfo: Conditional focused neural question answering with large-scale knowledge bases. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 800–810. Association for Computational Linguistics.https://doi.org/10.18653/v1/P16-1076. http://aclweb.org/anthology/P16-1076
Devlin J, Chang M, Lee K, Toutanova K (2018) BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018). arXiv: org/abs/1810.04805
Erling O, Mikhailov I (2009) RDF support in the virtuoso DBMS. In: Networked Knowledge—Networked Media—Integrating Knowledge Management, New Media Technologies and Semantic Systems, pp. 7–24.https://doi.org/10.1007/978-3-642-02184-8_2
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 00, pp. 770–778.https://doi.org/10.1109/CVPR.2016.90.
He X, Golub D (2016) Character-level question answering with attention. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1598–1607. Association for Computational Linguistics.https://doi.org/10.18653/v1/D16-1166. http://aclweb.org/anthology/D16-1166
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. CoRR abs/1412.6980 arXiv: org/abs/1412.6980
Lukov D, Fischer A, Lehmann J, Auer S (2017) Neural network-based question answering over knowledge graphs on word and character level. In: Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3–7, 2017, pp. 1211–1220.https://doi.org/10.1145/3038912.3052675
Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems—Volume 2, NIPS’13, pp. 3111–3119. Curran Associates Inc., USA.http://dl.acm.org/citation.cfm?id=2999792.2999959
Pennington J, Socher R, Manning C (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics.https://doi.org/10.3115/v1/D14-1162. http://aclweb.org/anthology/D14-1162
Qu Y, Liu J, Kang L, Shi Q, Ye D (2018) Question answering over freebase via attentive RNN with similarity matrix based CNN. CoRR abs/1804.03317v2. arXiv: org/abs/1804.03317v2
Reddy S, Lapata M, Steedman M (2014) Large-scale semantic parsing without question-answer pairs. TACL 2, 377–392.http://dblp.uni-trier.de/db/journals/tacl/tacl2.html#ReddyLS14
Ryan W, Lin S (2009) Channel Codes: Classical and Modern. Cambridge University Press, Cambridge.https://doi.org/10.1017/CBO9780511803253
Serban IV, García-Durán A, Gülçehre Ç, Ahn S, Chandar S, Courville AC, Bengio Y (2016) Generating factoid questions with recurrent neural networks: The 30m factoid question-answer corpus. CoRR abs/1603.06807 arXiv:org/abs/1603.06807
Türe F, Jojic O (2017) No need to pay attention: Simple recurrent neural networks work! In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9–11, 2017, pp. 2866–2872.https://aclanthology.info/papers/D17-1307/d17-1307
Uyar A, Aliyu FM (2015) Evaluating search features of google knowledge graph and bing satori: entity types, list searches and query interfaces. Online Inf Rev 39(2):197–213.https://doi.org/10.1108/OIR-10-2014-0257
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA, pp. 6000–6010.http://papers.nips.cc/paper/7181-attention-is-all-you-need
Xu K, Reddy S, Feng Y, Huang S, Zhao D (2016) Question answering on freebase via relation extraction and textual evidence. In: ACL (1). The Association for Computer Linguistics.http://dblp.uni-trier.de/db/conf/acl/acl2016-1.html#XuRFHZ16
Yih Wt, Chang MW, He X, Gao J (2015) Semantic parsing via staged query graph generation: Question answering with knowledge base. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1321–1331. Association for Computational Linguistics.https://doi.org/10.3115/v1/P15-1128. http://aclweb.org/anthology/P15-1128
Yin W, Yu M, Xiang B, Zhou B, Schütze H (2016) Simple question answering by attentive convolutional neural network. In: COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11–16, 2016, Osaka, Japan, pp. 1746–1756.http://aclweb.org/anthology/C/C16/C16-1164.pdf
Yu M, Yin W, Hasan KS, dos Santos CN, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30—August 4, Volume 1: Long Papers, pp. 571–581.https://doi.org/10.18653/v1/P17-1053
Acknowledgements
We would like to thank the anonymous reviewers for their valuable comments. Dr. Hao Wu is the corresponding author. Our work is partially supported by the National Key R&D Program of China under Grant (2018YFC0830705) and NSFC under Grant (U19B2020 and 61772074).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, X., Zang, H., Yu, X. et al. On improving knowledge graph facilitated simple question answering system. Neural Comput & Applic 33, 10587–10596 (2021). https://doi.org/10.1007/s00521-021-05762-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-021-05762-9