Abstract
Reinforcement learning methods, surviving the control difficulties of the unknown environment, are gaining more and more popularity recently in the autonomous robotics community. One of the possible difficulties of the reinforcement learning applications in complex situations is the huge size of the state-value- or action-value-function representation [2]. The case of continuous environment (continuous valued) reinforcement learning could be even complicated, as the state-value- or action-value-functions are turning into continuous functions. In this paper we suggest a way for tackling these difficulties by the application of SVD (Singular Value Decomposition) methods [3], [4], [15], [26].
On research leave from: Department of Information Technology, University of Miskolc, Miskolc-Egyetemváros, Miskolc, H-3515, Hungary
This research was partly supported by the Hungarian National Scientific Research Fund grant no: F 029904.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bellman, R. E.: Dynamic Programming. Princeton University Press, Princeton, NJ (1957)
Sutton, R. S., A. G. Barto: Reinforcement Learning: An Introduction, MIT Press, Cambridge (1998)
Yam, Y., Baranyi, P., Yang, C. T.: Reduction of Fuzzy Rule Base Via Singular Value Decomposition. IEEE Transaction on Fuzzy Systems. Vol.: 7, No. 2 (1999) 120–131.
Baranyi, P., Yam, Y.: Fuzzy rule base reduction. Chapter 7 of Fuzzy IF-THEN Rules in Computational Intelligence: Theory and Applications Eds., D. Ruan and E.E. Kerre, Kluwer (2000) 135–160
Watkins, C. J. C. H.: Learning from Delayed Rewards. Ph.D. thesis, Cambridge University, Cambridge, England (1989)
Appl, M.: Model-based Reinforcement Learning in Continuous Environments. Ph.D. thesis, Technical University of München, München, Germany, dissertation.de, Verlag im Internet (2000)
Wang, L.X.: Fuzzy Systems are Universal Approximators. Proceedings of the First IEEE Conference on Fuzzy Systems, San Diego (1992) 1163–1169
Castro, J.L.: Fuzzy Logic Controllers are Universal Approximators. IEEE Transaction on SMC, Vol.25, 4 (1995)
Horiuchi, T., Fujino, A., Katai, O., Sawaragi, T.: Fuzzy Interpolation-Based Q-learning with Continuous States and Actions. Proc. of the 5th IEEE International Conference on Fuzzy Systems, Vol.1 (1996) 594–600
Glorennec, P.Y., Jouffe, L.: Fuzzy Q-Learning. Proc. of the 6th IEEE International Conference on Fuzzy Systems (1997) 659–662
Berenji, H.R.: Fuzzy Q-Learning for Generalization of Reinforcement Learning. Proc. of the 5th IEEE International Conference on Fuzzy Systems (1996) 2208–2214
Bonarini, A.: Delayed Reinforcement, Fuzzy Q-Learning and Fuzzy Logic Controllers. In Herrera, F., Verdegay, J. L. (Eds.) Genetic Algorithms and Soft Computing, (Studies in Fuzziness, 8), Physica-Verlag, Berlin, D, (1996) 447–466
Baranyi, P., Várkonyi-Kóczy, A., Yam, Y., Patton, R.J., Michelberger, P., Sugiyama M.:SVD Based Reduction of TS Models. IEEE Trans. Industrial Electronics (accepted with minor revision)
Yen, J., Wang, L.: Simplifying Fuzzy Rule-based Models Using Orthogonal Transformation Methods. IEEE Trans. SMC, Vol 29: Part B, No. 1 (1999) 13–24
Yam, Y.: Fuzzy approximation via grid point sampling and singular value decomposition. IEEE Trans. SMC, Vol. 27 (1997) 933–951
Kóczy, L.T., Hirota, K.: Size Reduction by Interpolation in Fuzzy Rule Bases. IEEE Trans. SMC, vol. 27 (1997) 14–25
Tikk, D.: On nowhere denseness of certain fuzzy controllers containing prerestricted number of rules. Tatra Mountains Mathematical Publications vol. 16. (1999) 369–377
Lei, K., Baranyi, P., Yam, Y.: Complexity Minimalisation of Non-singleton Based Fuzzy-Neural Network. International Journal of Advanced Computational Intelligence, vol.4, no.4 (2000) 1–8
Baranyi, P., Yam, Y., Várlaki, P., Michelberger, P.: Singular Value Decomposition of Linguistically Defined Relations. Int. Jour. Fuzzy Systems, Vol. 2, No. 2, June (2000) 108–116
Song, F., Smith, S.M.: A Simple Based Fuzzy Logic Controller Rule Base Reduction Method. IEEE Int. Conf. System Man and Cybernetics (IEEE SMC’ 2000), Nashville, Tennessee, USA (2000) 3794–3798
Setnes, M., Hellendoorn, H.: Orthogonal Transforms for Ordering and Reduction of Fuzzy Rules. 9th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 2000), San Antonio, Texas (2000) 700–705
Sudkamp, T., Knapp, A., Knapp J.: A Greedy Approach to Rule Reduction in Fuzzy Models. IEEE Int. Conf. System Man and Cybernetics (IEEE SMC’2000), Nashville, Tennessee, USA (2000) 3716–3721
Baranyi, P., Yam, Y., Yang, C.T., Várlaki, P., Michelberger, P.: Generalised SVD Fuzzy Rule Base Complexity Reduction. International Journal of Advanced Computational Intelligence (accepted, to be printed in 2001)
Baranyi, P., Yam, Y.: Singular Value-Based Approximation with Non-Singleton Fuzzy Rule Base. 7th Int. Fuzzy Systems Association World Congress (IFSA’97) Prague (1997) 127–132
Baranyi, P., Yam, Y.: Singular Value-Based Approximation with Takagi-Sugeno Type Fuzzy Rule Base. 6th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE’97) Barcelona, Spain (1997) 265–270
Baranyi, P., Várkonyi-Kóczy, A.R., Yam, Y., Várlaki, P., Michelberger, P.: An Adaption Technique to SVD Reduced Rule Bases. IFSA 2001, Vancouver (accepted for presentation)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kovács, S. (2001). SVD Reduction in Continuos Environment Reinforcement Learning. In: Reusch, B. (eds) Computational Intelligence. Theory and Applications. Fuzzy Days 2001. Lecture Notes in Computer Science, vol 2206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45493-4_71
Download citation
DOI: https://doi.org/10.1007/3-540-45493-4_71
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42732-2
Online ISBN: 978-3-540-45493-9
eBook Packages: Springer Book Archive