Abstract
Analogical proportions are statements of the form ‘a is to b as c is to d’, formally denoted
. This means that the way a and b (resp. b and a) differ is the same as c and d (resp. d and c) differ, as revealed by their logical modeling. The postulates supposed to govern such proportions entail that when
holds, then seven permutations of a, b, c, d still constitute valid analogies. It can also be derived that
does not hold except if \(a=b\). From a machine learning perspective, this provides guidelines to build training sets of positive and negative examples. We then suggest improved methods to classify word-analogies and also to solve analogical equations. Viewing words as vectors in a multi-dimensional space, we depart from the traditional parallelogram view of analogy to adopt a purely machine-learning approach. In some sense, we learn a functional definition of analogical proportions without assuming any pre-existing formulas. We mainly use the logical properties of proportions to define our training sets and to design proper neural networks, approximating the hidden relations. Using a GloVe embedding, the results we get show high accuracy and improve state of the art on words analogy-solving problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
From now on, we use lower case letter a to denote the word or the vector. Moreover we alleviate the notation by writing a instead of \(\overrightarrow{a}\) and so on for b, c, d.
References
Ando, S., Lepage, Y.: Linguistic structure analysis by analogy: its efficiency. In: NLPRS, Phuket (1997)
Bayoudh, M., Prade, H., Richard, G.: Evaluation of analogical proportions through Kolmogorov complexity. Knowl.-Based Syst. 29, 20–30 (2012)
Bounhas, M., Prade, H., Richard, G.: Analogy-based classifiers for nominal or numerical data. Int. J. Approximate Reasoning 91, 36–55 (2017)
Hwang, S.J., Grauman, K., Sha, F.: Analogy-preserving semantic embedding for visual object categorization. In: Proceedings of 30th ICML, vol. 28, III-639-647 (2013)
Jing, L., Lu, Y., Gang, H., Sing Bing, K.: Visual attribute transfer through deep image analogy. ACM Trans. Graph. (Proceedings of Siggraph) 36(4), 120:1–120:15 (2017)
Langlais, P., Patry, A.: Translating unknown words by analogical learning. In: Joint Conference on Empirical Method in NLP (EMNLP) and Conference on Computational Natural Langauge Learning (CONLL), Prague, pp. 877–886 (2007)
Lepage, Y.: Analogy and formal languages. Electr. Notes Theor. Comput. Sci. 53, 189–191 (2001)
Lepage, Y., Denoual, E.: Purest ever example-based machine translation: detailed presentation and assessment. Mach. Transl. 19(3–4), 251–282 (2005)
Lepage, Y., Migeot, J., Guillerm, E.: A measure of the number of true analogies between chunks in Japanese. In: Vetulani, Z., Uszkoreit, H. (eds.) LTC 2007. LNCS (LNAI), vol. 5603, pp. 154–164. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04235-5_14
Levy, O., Goldberg, Y.: Dependency-based word embeddings. In: Proceedings of 52nd Annual Meeting of Association Computational Linguistics (Vol 2: Short Papers), pp. 302–308 (2014)
Levy, O., Goldberg, Y., Dagan, I.: Improving distributional similarity with lessons learned from word embeddings. Trans. Ass. Comput. Ling. 3, 211–225 (2015)
Lu, H., Wu, Y., Holyoak, K.H.: Emergence of analogy from relation learning. Proc. Nat. Acad. Sci. 116, 4176–4181 (2019)
Miclet, L., Prade, H.: Handling analogical proportions in classical logic and fuzzy logics settings. In: Sossai, C., Chemello, G. (eds.) ECSQARU 2009. LNCS (LNAI), vol. 5590, pp. 638–650. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02906-6_55
Mikolov, T., Chen, K., Corrado, G.S., Dean, J.: Efficient estimation of word representations in vector space. CoRR, abs/1301.3781 (2013)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C. et al., (eds.) Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119. Curran Associates Inc. (2013)
Mikolov, T., Yih, W., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746–751 (2013)
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP), vol. 14, pp. 1532–1543. Association for Computational Linguistics, January 2014
Prade, H., Richard, G.: From analogical proportion to logical proportions. Logica Univers. 7, 441–505 (2013)
Reitman, W.R.: Cognition and thought. an information processing approach. Psych. in the Schools, vol. 3, no. 2 (1965)
Rumelhart, D.E., Abrahamson, A.A.: A model for analogical reasoning. Cogn. Psychol. 5, 1–28 (2005)
Stroppa, N., Yvon, F.: Analogical learning and formal proportions: Definitions and methodological issues. Technical Report D004, ENST-Paris (2005)
Turney, P.D.: Similarity of semantic relations. Comput. Linguist. 32(3), 379–416 (2006)
Turney, P.D.: The latent relation mapping engine: algorithm and experiments. JAIR 33, 615–655 (2008)
Turney, P.D.: A uniform approach to analogies, synonyms, antonyms, and associations. In: Proceedings of 22nd International Conference on Computational Linguistics - vol. 1 (COLING 2008), pp. 905–912. Association for Computational Linguistics (2008)
Veale, T.: Re-representation and creative analogy: a lexico-semantic perspective. New Gener. Comput. 24(3), 223–240 (2006)
Yvon, F., Stroppa, N., Delhay, A., Miclet, L.: Solving analogical equations on words. Technical report, Ecole Nationale Supérieure des Télécommunications (2004)
Acknowledgements
This work was partially supported by ANR-11-LABX-0040-CIMI (Centre International de Mathématiques et d’Informatique) within the program ANR-11-IDEX-0002-02, project ISIPA.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Lim, S., Prade, H., Richard, G. (2019). Solving Word Analogies: A Machine Learning Perspective. In: Kern-Isberner, G., Ognjanović, Z. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2019. Lecture Notes in Computer Science(), vol 11726. Springer, Cham. https://doi.org/10.1007/978-3-030-29765-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-29765-7_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29764-0
Online ISBN: 978-3-030-29765-7
eBook Packages: Computer ScienceComputer Science (R0)