Abstract
This paper propose to investigate a better way to apply Transfer Learning (TL) between agents to speed up the Q-learning Reinforcement Learning algorithm and combines Case-Based Reasoning (CBR) and Heuristically Accelerated Reinforcement Learning (HARL) techniques.
The experiments were made comparing differents approaches of Transfer Learning were actions learned in the acrobot problem can be used to speed up the learning of the policies of stability for Robocup 3D.
The results confirm that the same Transfer Learning information can show differents results, depending how is applied.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7(1), 39–59 (1994)
Bianchi, R.A.C., Ribeiro, C.H.C., Costa, A.H.R.: Heuristically Accelerated Q-Learning: a new approach to speed up reinforcement learning. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 245–254. Springer, Heidelberg (2004)
Bianchi, R.A.C., Ros, R., de Mántaras, R.L.: Improving reinforcement learning by using case based heuristics. In: McGinty, L., Wilson, D.C. (eds.) ICCBR 2009. LNCS, vol. 5650, pp. 75–89. Springer, Heidelberg (2009)
Boedecker, J., Dorer, K., Rollmann, M., Xu, Y., Xue, F., Buchta, M., Vatankhah, H.: Spark 3d simuation system (2010)
Caruana, R.: Learning many related tasks at the same time with backpropagation. In: Advances in Neural Information Processing Systems, vol. 7, pp. 657–664. Morgan Kaufmann, San Francisco (1995)
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
DARPA. Transfer learning proposer information pamphlet (2005)
de Mántaras, R.L., McSherry, D., Bridge, D., Leake, D., Smyth, B., Craw, S., Faltings, B., Maher, M.L., Cox, M.T., Forbus, K., Keane, M., Aamodt, A., Watson, I.: Retrieval, reuse, revision and retention in case-based reasoning. Knowl. Eng. Rev. 20(3), 215–240 (2005)
DeJong, G., Spong, M.W.: Swinging up the acrobot: An example of intelligent control. In: Proceedings of the American Control Conference, pp. 2158–2162 (1994)
Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E., Matsubara, H.: RoboCup: A Challenge AI Problem. AI Magazine 18(1), 73–85 (1997)
Littman, M.L., Szepesvári, C.: A generalized reinforcement learning model: convergence and applications. In: Proceedings of the 13th International Conference on Machine Learning (ICML 1996), pp. 310–318 (1996)
Mitchell, T.: Machine Learning. McGraw-Hill Education (ISE Editions), New York (1997)
Obst, O., Rollmann, M.: SPARK – A Generic Simulator for Physical Multiagent Simulations. Computer Systems Science and Engineering 20(5), 347–356 (2005)
Ros, R.: Action Selection in Cooperative Robot Soccer using Case-Based Reasoning. PhD thesis, Universitat Autònoma de Barcelona, Barcelona (2008)
Ros, R., Arcos, J.L., de Mántaras, R.L., Veloso, M.: A case-based approach for coordinated action selection in robot soccer. Artificial Intelligence 173(9-10), 1014–1039 (2009)
Spiegel, M.R.: Statistics. McGraw-Hill, New York (1998)
Sutton, R.S.: Generalization in reinforcement learning: Successful examples using sparse coarse coding. In: Advances in Neural Information Processing Systems, vol. 8, pp. 1038–1044. MIT Press, Cambridge (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Taylor, M.E., Jong, N.K., Stone, P.: Transferring instances for model-based reinforcement learning. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 488–505. Springer, Heidelberg (2008)
Taylor, M.E., Kuhlmann, G., Stone, P.: Autonomous transfer for reinforcement learning. In: The Seventh International Joint Conference on Autonomous Agents and Multiagent Systems (May 2008)
Taylor, M.E., Stone, P.: Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research 10(1), 1633–1685 (2009)
Thorndike, E.L., Woodworth, R.S.: The influence of improvement in one mental function upon the efficiency of other functions. Psychological Review 8, 247–261 (1901)
Thrun, S.: Is learning the n-th thing any easier than learning the first? In: Advances in Neural Information Processing Systems, pp. 640–646. The MIT Press, Cambridge (1996)
Thrun, S., Mitchell, T.M.: Learning one more thing. In: IJCAI 1995: Proceedings of the 14th International Joint Conference on Artificial Intelligence, pp. 1217–1223. Morgan Kaufmann Publishers Inc., San Francisco (1995)
Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD thesis, University of Cambridge (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Celiberto Junior, L.A., Matsuura, J.P. (2011). Investigation in Transfer Learning: Better Way to Apply Transfer Learning between Agents. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2011. Lecture Notes in Computer Science(), vol 6871. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23199-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-23199-5_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23198-8
Online ISBN: 978-3-642-23199-5
eBook Packages: Computer ScienceComputer Science (R0)