Abstract
Engineers and researchers are paying more attention to reinforcement learning (RL) as a key technique for realizing computational intelligence such as adaptive and autonomous decentralized systems. In general, it is not easy to put RL into practical use. In previous research, Nagayoshi et al. have proposed an adaptive co-construction method of state and action spaces. However, the co-construction method needs two parameters for sufficiency of the number of learning opportunities. These parameters are difficult to set. In this paper, first we propose an entropy-guided adaptive co-construction method with and index using the entropy instead of the parameters for sufficiency of the number of learning opportunities. Then, the performance of the proposed method and the efficiency of interactions between state and action spaces were confirmed through computational experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning. A Bradford Book. MIT Press (1998)
Sutton, R.S.: Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding. In: Advances in Neural Information Processing Systems: Proc. of the 1995 Conf., pp. 1038–1044 (1996)
Doya, K.: Temporal Difference Learning in Continuous Time and Space. In: Proc. of the 1995 Conf. on Advances in Neural Information Processing Systems, pp. 1073–1079 (1996)
Murao, H., Kitamura, S.: QLASS: An enhancement of Q-learning to generate state space adaptively. In: Proc. of Fourth European Conf. on Artificial Life, ECAL 1997 (1997)
Nagayoshi, M., Murao, H., Tamaki, H.: A State Space Filter for Reinforcement Learning. In: Proc. of the Eleventh International Symposium on Artificial Life and Robotics, pp. 615–618 (2006)
Morimoto, J., Doya, K.: Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. Robotics and Autonomous System 36, 35–51 (2001)
Nagayoshi, M., Murao, H., Tamaki, H.: A Reinforcement Learning with Switching Controllers for Continuous Action Space. Artificial Life and Robotics 15(1), 97–100 (2010)
Nagayoshi, M., Murao, H., Tamaki, H.: Developing Reinforcement Learning for Adaptive Co-construction of Continuous High-dimensional State and Action Spaces. Artificial Life and Robotics 17(2), 204–210 (2012)
Appel, G.: Technical Analysis Power Tools for Active Investors. Financial Times Prentice Hall (2005)
Kimura, H., Kobayashi, S.: An Analysis of Actor/Critic Algorithms using Eligibility Traces: Reinforcement Learning with Imperfect Value Function. In: Proc. of the Fifteenth International Conference on Machine Learning, pp. 278–286 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Nagayoshi, M., Murao, H., Tamaki, H. (2014). An Entropy-Guided Adaptive Co-construction Method of State and Action Spaces in Reinforcement Learning. In: Loo, C.K., Yap, K.S., Wong, K.W., Teoh, A., Huang, K. (eds) Neural Information Processing. ICONIP 2014. Lecture Notes in Computer Science, vol 8834. Springer, Cham. https://doi.org/10.1007/978-3-319-12637-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-12637-1_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12636-4
Online ISBN: 978-3-319-12637-1
eBook Packages: Computer ScienceComputer Science (R0)