Abstract
Semantic similarity between words has now become a popular research problem to tackle in natural language processing (NLP) field. Word embedding have been demonstrated progress in measuring word similarity recently. However, limited to the distributional hypothesis, basic embedding methods generally have drawbacks in nature. One of the limitations is that word embeddings are usually by predicting a target word in its local context, leading to only limited information being captured. In this paper, we propose a novel transferred vectors approach to compute word semantic similarity. Transferred vectors are obtained via a reasonable combination of the source word and its nearest neighbors on semantic level. We conduct experiments on popular both English and Chinese benchmarks for measuring word similarity. The experiment results demonstrate that our method outperforms previous state-of-the-art by a large margin.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mihalcea, R., Corley, C., Strapparava, C.: Corpus-based and knowledge-based measures of text semantic similarity. In: AAAI, pp. 775–780 (2006)
Islam, A., Inkpen, D.: ACM Trans. Knowl. Discov. Data, 2(2), Article 10 (2008)
Gao, J., Nie, J.-Y., Xun, E., Zhang, J., Zhou, M., Huang, C.: Improving query translation for cross-language information retrieval using statistical models. In: International ACM SIGIR Conference on Research & Development in Information Retrieval (2001)
Hassan, H., Hassan, A., Emam, O.: Unsupervised information extraction approach using graph mutual reinforcement. In: Conference on Empirical Methods in Natural Language Processing (2006)
Cambria, E., Schuller, B., Xia, Y., Havasi, C.: New avenues in opinion mining and sentiment analysis. Intell. Syst. IEEE 28(2), 15–21 (2013)
Li, Y., McLean, D., Bandar, Z.A., O’shea, J.D., Crockett, K.: Sentence similarity based on semantic nets and corpus statistics. IEEE Trans. Knowl. Data Eng. 18(8), 1138–1150 (2006)
Luong, M., Socher, R., Manning, C.: Better word representations with recursive neural networks for morphology. In: CoNLL-2013, pp. 104–113 (2013)
Turian, J., Ratinov, L., Bengio, Y.: Word representations: a simple and general method for semi-supervised learning. In: ACL 2010 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 384–394 (2010)
Panchenko, A., Morozova, O., Naets, H.: A semantic similarity measure based on lexico-syntactic patterns. In: Conference on Natural Language Processing (KONVENS 2012), Vienna (Austria), pp. 174–178 (2012)
Wu, Y., Li, W.: Chinese word similarity measurement. NLPCC-ICCPOL 2016 shared task 3. In: Proceedings of NLPCC 2016 (2016)
Fellbaum, C.: WordNet. Wiley Online Library (1998)
Tian, J.L., Zhao, W.: Words similarity algorithm based on Tongyici Cilin in semantic web adaptive learning system. J. Jilin Univ. 28(06), 602–608 (2010)
Mei, J.J., Zhu, Y.M., et al.: Tongyici Cilin. Shanghai Lexicon Publishing Company, Shanghai (1983)
Dong, Z., Dong, Q.: HowNet and the Computation of Meaning, pp. 85–95. World Scientific, Singapore (2006)
Liu, Q., Li, S.: Word similarity computing based on HowNet. Comput. Linguist. Chin. Lang. Process. 7(2), 59–76 (2002)
Miller, G.A., Charles, W.G.: Contextual correlates of semantic similarity. Lang. Cogn. Process. 6, 1–28 (1991)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013)
Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of the Eighth Annual Conference of the Cognitive Science Society, Amherst, MA, pp. 1–12 (1986)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, pp. 1–12 (2013)
Mikolov, T., Sutskever, I., et al.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NiPS, pp. 3111–3119 (2013)
Levy, Q., Goldberg, Y.: Neural word embedding as implicit matrix factorization. In: Advances in Neural Information Processing Systems, pp. 2177–2185 (2014)
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning (ICML 2014), ICML 2014, pp. 1188–1196 (2014)
Iacobacci, I., Pilehvar, M.T., Navigli, R.: SensEmbed: learning sense embeddings for word and relational similarity. In: Proceeding of ACL, pp. 95–105 (2015)
Chen, W., Zhang, Y., Zhang, M.: Feature embedding for dependency parsing. In: Proceedings of COLING 2014, The 25th International Conference on Computational Linguistics: Technical Papers, pp. 816–826 (2014)
Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)
Liu, K.: Research on Chinese FrameNet construction and application technologies. J. Chin. Inf. Process. 6, 47 (2011)
Rubenstein, H., Goodenough, J.B.: Contextual correlates of synonymy. Commun. ACM 8(10), 627–633 (1965)
Huang, E.H., Socher, R., Manning, C.D., Ng, A.Y.: Improving word representations via global context and multiple word prototypes. In: Annual Meeting of the Association for Computational Linguistics (ACL) (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Li, C., Ma, T., Zhou, Y., Cheng, J., Xu, B. (2017). Measuring Word Semantic Similarity Based on Transferred Vectors. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10637. Springer, Cham. https://doi.org/10.1007/978-3-319-70093-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-70093-9_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70092-2
Online ISBN: 978-3-319-70093-9
eBook Packages: Computer ScienceComputer Science (R0)