Abstract
Reasoning systems with too simple a model of the world and human intent are unable to consider potential negative side effects of their actions and modify their plans to avoid them (e.g., avoiding potential errors). However, hand-encoding the enormous and subtle body of facts that constitutes common sense into a knowledge base has proved too difficult despite decades of work. Distributed semantic vector spaces learned from large text corpora, on the other hand, can learn representations that capture shades of meaning of common-sense concepts and perform analogical and associational reasoning in ways that knowledge bases are too rigid to perform, by encoding concepts and the relations between them as geometric structures. These have, however, the disadvantage of being unreliable, poorly understood, and biased in their view of the world by the source material. This chapter will discuss how these approaches may be brought together in a way that combines the best properties of each for understanding the world and human intentions in a richer way.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Mané, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565.
Ba, J. L., Kiros, J. R., and Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450.
Blouw, P and Eliasmith, C. (2005) A neurally plausible encoding of word order information into a semantic vector space. 35th Annual conference of the cognitive science society Vol. 1910.
Dash, D., Voortman, M., and De Jongh, M. (2013, August). Sequences of Mechanisms for Causal Reasoning in Artificial Intelligence. In IJCAI
Deerwester, S., et al, Improving Information Retrieval with Latent Semantic Indexing, Proceedings of the 51st Annual Meeting of the American Society for Information Science 25, 1988, pp. 36–40.
Dietterich, T. G., and Horvitz, E. J. (2015). Rise of concerns about AI: reflections and directions. Communications of the ACM, 58(10), 38-40.
Faruqui, M., Dodge, J., Jauhar, S. K., Dyer, C., Hovy, E., and Smith, N. A. (2014). Retrofitting word vectors to semantic lexicons. arXiv preprint arXiv:1411.4166.
Hawkins, J., and Blakeslee, S. (2007). On intelligence. Macmillan.
Hayes, P. J. (1978). The naive physics manifesto. Institut pour les études sémantiques et cognitives/Université de Genève.
Hinton, G. E. (1984). Distributed representations.
Hofstadter, D. (1985). Metamagical themas: Questing for the essence of mind and pattern. Basic books.
Hofstadter, D, and Sander, E. Surfaces and Essences. Basic Books, 2013.
Huth, A. G., Nishimoto, S., Vu, A. T., and Gallant, J. L. (2012). A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76(6), 1210-1224.
Kanerva, P. (1988). Sparse distributed memory. MIT press.
Kiros, R., Zhu, Y., Salakhutdinov, R. R., Zemel, R., Urtasun, R., Torralba, A., and Fidler, S. (2015). Skip-thought vectors. In Advances in neural information processing systems (pp. 3294-3302).
Leech, R., Mareschal, D., and Cooper, R. P. (2008). Analogy as relational priming: A developmental and computational perspective on the origins of a complex cognitive skill. Behavioral and Brain Sciences, 31(04), 357-378.
Lenat, D. B., Prakash, M., & Shepherd, M. (1985). CYC: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks. AI magazine, 6(4), 65.
Levy, O., and Goldberg, Y. (2014). Dependency-Based Word Embeddings. In ACL (2) (pp. 302-308).
Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., and Liu, S. (2015). Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379.
Neelakantan, A., Roth, B., and Mc-Callum, A. (2015, March). Compositional vector space models for knowledge base inference. In 2015 AAAI Spring Symposium Series.
Reed, S. E., Zhang, Y., Zhang, Y., and Lee, H. (2015). Deep visual analogy-making. In Advances in Neural Information Processing Systems (pp. 1252-1260).
Rei, M., and Briscoe, T. (2014, June). Looking for Hyponyms in Vector Space. In CoNLL (pp. 68-77).
Rissman, J., and Wagner, A. D. (2012). Distributed representations in memory: insights from functional brain imaging. Annual review of psychology, 63, 101.
Rothe, S., and Schütze, H. (2015). Autoextend: Extending word embeddings to embeddings for synsets and lexemes. arXiv preprint arXiv:1507.01127.
Russell, S. (2014, November 14). Of Myths And Moonshine. Retrieved from edge.org/conversation/jaron_lanier-the-myth-of-ai
Sadeghi, F., Zitnick, C. L., and Farhadi, A. (2015). Visalogy: Answering visual analogy questions. In Advances in Neural Information Processing Systems (pp. 1882-1890).
Speer, R., Havasi, C., and Lieberman, H. (2008, July). AnalogySpace: Reducing the Dimensionality of Common Sense Knowledge. In AAAI (Vol. 8, pp. 548-553).
Turney, P. D. (2006). Similarity of semantic relations. Computational Linguistics, 32(3), 379-416.
Upchurch, P., Snavely, N., and Bala, K. (2016). From A to Z: Supervised Transfer of Style and Content Using Deep Neural Network Generators. arXiv preprint arXiv:1603.02003.
Vosniadou, S., and Ortony, A. (1989). Similarity and analogical reasoning. Cambridge University Press.
Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014, October). Knowledge Graph and Text Jointly Embedding. In EMNLP (pp. 1591-1601).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Summers-Stay, D. (2017). Semantic Vector Spaces for Broadening Consideration of Consequences. In: Lawless, W., Mittu, R., Sofge, D., Russell, S. (eds) Autonomy and Artificial Intelligence: A Threat or Savior?. Springer, Cham. https://doi.org/10.1007/978-3-319-59719-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-59719-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59718-8
Online ISBN: 978-3-319-59719-5
eBook Packages: Computer ScienceComputer Science (R0)