Abstract
The adaptation of temporal differences method TD(λ> 0) to reinforcement learning algorithms with fuzzy approximation of action-value function is proposed. Eligibility traces are updated using the normalized degree of activation of fuzzy rules. The two types of fuzzy reinforcement learning algorithm are formulated: with discrete and with continuous action values. These new algorithms are practically tested in the control of two typical models of continuous object, like ball-beam and cart-pole system. The achievement results are compared with two popular reinforcement learning algorithms with CMAC and table approximation of action-value function.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike adaptive elements that can solve difficult learning problem. IEEE Trans. SMC 13, 834–847 (1983)
Bonarini, A., Lazaric, A., Montrone, F., Restelli, M.: Reinforcement distribution in Fuzzy Q-learning. Fuzzy Sets and Systems 160, 1420–1443 (2009)
Cichosz, P.: Learning systems. WNT, Warsaw (2000) (in Polish)
Gu, D., Hu, H.: Accuracy based fuzzy Q-learning for robot behaviours. In: Proc. of the IEEE Int. Conf. on Fuzzy Systems, vol. 3, pp. 1455–1460 (2004)
Min, H., Zeng, J., Luo, R.: Fuzzy CMAC with automatic state partition for reinforcement learning. In: Proc. of the First ACM/SIGEVO Summit on Genetic and Evolutionary Computation, pp. 421–428 (2009)
Nguyen, M.N., Shi, D., Quek, C.: Self-Organizin Gaussian fuzzy CMAC with Truth Value Restriction. In: Proc. of the Third Int. Conf. on Information Technology and Applications (ICITA 2005), vol. 2, pp. 185–190 (2005)
Shi, D., Harkisanka, A., Quek, C.: CMAC with Fuzzy Logic Reasoning. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 898–903. Springer, Heidelberg (2004)
Sutton, R.S.: Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding. Advances in Neural information Processing Systems 8, 1038–1044 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement learning: An Introduction. MIT Press, Cambridge (1998)
Theodoridis, T., Hu, H.: The Fuzzy Sars’a’(λ) Learning Approach Applied to a Strategic Route Learning Robot Behaviour. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 1767–1772 (2006)
Watkins, C.J.C.H.: Learning from delayed Rewards. PhD thesis, Cambridge University, Cambridge, England (1989)
Wellstead, P.E.: Introduction to Physical System Modelling, Control System Principles (2000)
Xu, X., Hu, D., He, H.: Accelerated Reinforcement learning control using modified CMAC neural networks. In: Proc. of the Ninth Int. Conf. on Neural Information Processing, vol. 5, pp. 2575–2578 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zajdel, R. (2010). Fuzzy Q(λ)-Learning Algorithm. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2010. Lecture Notes in Computer Science(), vol 6113. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13208-7_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-13208-7_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13207-0
Online ISBN: 978-3-642-13208-7
eBook Packages: Computer ScienceComputer Science (R0)