Abstract
Exploitation-oriented Learning (XoL) is a novel approach to goal-directed learning from interaction. Though reinforcement learning is much more focus on the learning and can gurantee the optimality in Markov Decision Processes (MDPs) environments, XoL aims to learn a rational policy, whose expected reward per an action is larger than zero, very quickly. We know PS-r* that is one of the XoL methods. It can learn an useful rational policy that is not inferior to a random walk in Partially Observed Markov Decision Processes (POMDPs) environments where the number of types of a reward is one. However, PS-r* requires O(MN 2) memories where N and M are the numbers of types of a sensory input and an action.In this paper, we propose PS-r# that can learn an useful rational policy in the POMDPs environments by O(MN) memories. We confirm the effectiveness of PS-r# in numerical examples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abbeel, P., Ng, A.Y.: Exploration and apprenticeship learning in reinforcement learning. In: Proc. of 22th International Conference on Machine Learning, pp. 1–8 (2005)
Chrisman, L.: Reinforcement Learning with perceptual aliasing: The Perceptual Distinctions Approach. In: Proc. of 10th National Conference on Artificial Intelligence, pp. 183–188 (1992)
Kimura, H., Yamamura, M., Kobayashi, S.: Reinforcement Learning by Stochastic Hill Climbing on Discounted Reward. In: Proc. of 12th International Conference on Machine Learning, pp. 295–303 (1995)
Merrick, K., Maher, M.L.: Motivated Reinforcement Learning for Adaptive Characters in Open-Ended Simulation Games. In: Proc. of the International Conference on Advanced in Computer Entertainment Technology, pp. 127–134 (2007)
McCallum, R.A.: Instance-Based Utile Distinctions for Reinforcement Learning with Hidden State. In: Proc. of 12th International Conference on Machine Learning, pp. 387–395 (1995)
Miyazaki, K., Yamamura, M., Kobayashi, S.: On the Rationality of Profit Sharing in Reinforcement Learning. In: Proc. of 3rd International Conference on Fuzzy Logic, Neural Nets and Soft Computing, pp. 285–288 (1994)
Miyazaki, K., Kobayashi, S.: Learning Deterministic Policies in Partially Observable Markov Decision Processes. In: Proc. of 5th International Conference on Intelligent Autonomous System, pp. 250–257 (1998)
Miyazaki, K., Kobayashi, S.: Reinforcement Learning for Penalty Avoiding Policy Making. In: Proc. of the 2000 IEEE International Conference on Systems, Man and Cybernetics, pp. 206–211 (2000)
Miyazaki, K., Kobayashi, S.: An Extension of Profit Sharing to Partially Observable Markov Decision Processes: Proposition of PS-r* and its Evaluation. Journal of the Japanese Society for Artificial Intelligence 18(5), 286–296 (2003) (in Japanese)
Miyazaki, K., Kobayashi, S.: A Reinforcement Learning System for Penalty Avoiding in Continuous State Spaces. Journal of Advanced Computational Intelligence and Intelligent Informatics 11(6), 668–676 (2007)
Ng, A.Y., Russell, S.J.: Algorithms for Inverse Reinforcement Learning. In: Proc. of 17th International Conference on Machine Learning, pp. 663–670 (2000)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, A Bradford Book. MIT Press, Cambridge (1998)
Williams, R.J.: Simple Statistical Gradient Following Algorithms for Connectionist Reinforcement Learning. Machine Learning 8, 229–256 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Miyazaki, K., Kobayashi, S. (2008). Proposal of Exploitation-Oriented Learning PS-r# . In: Fyfe, C., Kim, D., Lee, SY., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2008. IDEAL 2008. Lecture Notes in Computer Science, vol 5326. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88906-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-88906-9_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88905-2
Online ISBN: 978-3-540-88906-9
eBook Packages: Computer ScienceComputer Science (R0)