Abstract
In this paper, we propose a dynamic allocation method of basis functions, an Allocation/Elimination Gaussian Softmax Basis Function Network (AE-GSBFN), that is used in reinforcement learning. AE-GSBFN is a kind of actor-critic method that uses basis functions. This method can treat continuous high-dimensional state spaces, because basis functions required only for learning are dynamically allocated, and if an allocated basis function is identified as redundant, the function is eliminated. This method overcomes the curse of dimensionality and avoids a fall into local minima through the allocation and elimination processes. To confirm the effectiveness of our method, we used a maze task to compare our method with an existing method, which has only an allocation process. Moreover, as learning of continuous high-dimensional state spaces, our method was applied to motion control of a humanoid robot. We demonstrate that the AE-GSBFN is capable of providing better performance than the existing method.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Albus, J.S.: Brains, Behavior, and Robotics. Byte Books (1981)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Morimoto, J., Doya, K.: Reinforcement learning of dynamic motor sequence: Learning to stand up. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, pp. 1721–1726 (1998)
Samejima, K., Omori, T.: Adaptive state space formation method for reinforcement learning. In: International Conference on Neural Information Processing, pp. 251–255 (1998)
Takahashi, Y., Asada, M., Hosoda, K.: Reasonable performance in less learning time by real robot based on incremental state space segmentation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1518–1524 (1996)
Moore, A.W., Atkeson, C.G.: The parti-game algorithm for variable resolution reinforcement learning in multidimensional state space. Machine Learning 21, 199–234 (1995)
Gullapalli, V.: A stochastic reinforcement learning algorithm for learning real-valued functions. Neural Networks 3, 671–692 (1990)
Morimoto, J., Doya, K.: Learning dynamic motor sequence in high-dimensional state space by reinforcement learning — learning to stand up —. IEICE J82-D-II, 2118–2131 (1999)
Kondo, T., Ito, K.: A proposal of an on-line evolutionary reinforcement learning. In: 13th Autonomous Distributed Symposium (2001) (in Japanese)
Smith, R. (Open Dynamics Engine), http://opende.sourceforge.net/ode.html
Fritzke, B.: Growing self-organizing networks — why? In: European Symposium on Artificial Neural Networks, pp. 61–72 (1996)
Marsland, S., Shapiro, J., Nehmzow, U.: A self-organizing network that grows when required. Neural Networks 15, 1041–1058 (2002)
Morimoto, J., Doya, K.: Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. In: International Conference on Machine Learning, pp. 623–630 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Iida, S., Kuwayama, K., Kanoh, M., Kato, S., Itoh, H. (2004). A Dynamic Allocation Method of Basis Functions in Reinforcement Learning. In: Webb, G.I., Yu, X. (eds) AI 2004: Advances in Artificial Intelligence. AI 2004. Lecture Notes in Computer Science(), vol 3339. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30549-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-540-30549-1_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24059-4
Online ISBN: 978-3-540-30549-1
eBook Packages: Computer ScienceComputer Science (R0)