Abstract
Addressing the sparse reward problem in Deep Reinforcement Learning (DRL) using human supplied external knowledge or reasoning is a common practice. Such external knowledge and reasoning should not be so complete that a DRL agent does not almost need to perform any exploration questioning its utility. Non-analytical Reasoning could shape an agent’s actions sufficiently yet take away minimal credit from the DRL exploration process. We generalize the solution approaches to Non-analytical Reasoning Assisted Deep Reinforcement Learning and present an example solution to “Montezuma’s Revenge,” a notorious Atari game, applying such reasoning.
This material is based upon work supported by the National Science Foundation under Award No. OIA-1946391. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environment: an evaluation platform for general agents. J. Artif. Intell. Res. 47, 253–279 (2013)
Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., Efros, A.A.: Large-scale study of curiosity-driven learning. arXiv preprint arXiv:1808.04355 (2018)
Colin, T.R., Belpaeme, T.: Reinforcement learning and insight in the artificial pigeon. In: 41st Annual Meeting of the Cognitive Science Society (CogSci 2019), pp. 1533–1539. Cognitive Science Society (2019)
Esteves, J.J.A., Boubendir, A., Guillemin, F., Sens, P.: A heuristically assisted deep reinforcement learning approach for network slice placement. IEEE Trans. Netw. Serv. Manag. (2021)
IntelLabs: Intellabs/coach: Reinforcement learning coach by intel ai lab enables easy experimentation with state of the art reinforcement learning algorithms. https://github.com/IntelLabs/coach
Kaplan, C.A., Simon, H.A.: In search of insight. Cogn. Psychol. 22(3), 374–419 (1990)
McCrea, S.M.: Intuition, insight, and the right hemisphere: emergence of higher sociocognitive functions. Psychol. Res. Behav. Manag. (2010)
Mnih, V., et al.: Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
Ng, A.Y., Harada, D., Russell, S.: Policy invariance under reward transformations: theory and application to reward shaping. In: Icml, vol. 99, pp. 278–287 (1999)
Berner, C., et al.: Dota 2 with large scale deep reinforcement learning (2019). OpenAI
Romanycia, M.H., Pelletier, F.J.: What is a heuristic? Comput. Intell. 1(1), 47–58 (1985)
Salimans, T., Chen, R.: Learning montezuma’s revenge from a single demonstration. CoRR abs/1812.03381 (2018). http://arxiv.org/abs/1812.03381
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (2018)
Zander, T., Öllinger, M., Volz, K.G.: Intuition and insight: two processes that build on each other or fundamentally differ? Front. Psychol. 7, 1395 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Schonefeld, J., Karim, M. (2022). Non-analytical Reasoning Assisted Deep Reinforcement Learning. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-06527-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06526-2
Online ISBN: 978-3-031-06527-9
eBook Packages: Computer ScienceComputer Science (R0)