Abstract
In this paper we investigate and develop a real-world reinforcement learning approach to autonomously recharge a humanoid Nao robot [1]. Using a supervised reinforcement learning approach, combined with a Gaussian distributed states activation, we are able to teach the robot to navigate towards a docking station, and thus extend the duration of autonomy of the Nao by recharging. The control concept is based on visual information provided by naomarks and six basic actions. It was developed and tested using a real Nao robot within a home environment scenario. No simulation was involved. This approach promises to be a robust way of implementing real-world reinforcement learning, has only few model assumptions and offers faster learning than conventional Q-learning or SARSA.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Nao academics edition: medium-sized humanoid robot developed by Aldebaran Robotics, http://www.aldebaran-robotics.com/
Conn, K., Peters, R.A.: Reinforcement learning with a supervisor for a mobile robot in a real-world environment. In: International Symposium on Computational Intelligence in Robotics and Automation, CIRA, pp. 73–78. IEEE, Los Alamitos (2007)
Dorigo, M., Colombetti, M.: Robot shaping: An experiment in behavior engineering (intelligent robotics and autonomous agents). The MIT Press, Cambridge (1997)
Foster, D., Morris, R., Dayan, P.: A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10(1), 1–16 (2000)
Ghory, I.: Reinforcement learning in board games. Tech. rep., Department of Computer Science, University of Bristol (2004)
Ito, K., Fukumori, Y., Takayama, A.: Autonomous control of real snake-like robot using reinforcement learning; abstraction of state-action space using properties of real world. In: Palaniswami, M., Marusic, S., Law, Y.W. (eds.) Proceedings of the 3rd International Conference on Intelligent Sensors, Sensor Networks and Information, ISSNIP, pp. 389–394. IEEE, Los Alamitos (2007)
Kietzmann, T.C., Riedmiller, M.: The neuro slot car racer: Reinforcement learning in a real world setting. In: International Conference on Machine Learning and Applications, ICMLA, pp. 311–316. IEEE, Los Alamitos (2009)
The KSERA project (Knowledgeable SErvice Robots for Aging), http://ksera.ieis.tue.nl/
Louloudi, A., Mosallam, A., Marturi, N., Janse, P., Hernandez, V.: Integration of the humanoid robot Nao inside a smart home: A case study. In: Proceedings of the Swedish AI Society Workshop (SAIS). Linköping Electronic Conference Proceedings, vol. 48, pp. 35–44. Uppsala University, Linköping University Electronic Press (2010)
Muse, D., Wermter, S.: Actor-Critic learning for Platform-Independent robot navigation. Cognitive Computation 1(3), 203–220 (2009)
Provost, J., Kuipers, B.J., Miikkulainen, R.: Self-organizing perceptual and temporal abstraction for robot reinforcement learning. In: AAAI Workshop on Learning and Planning in Markov Processes (2004)
The RobotDoC collegium: The Marie Curie doctoral training network in developmental robotics, http://robotdoc.org/
Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction (adaptive computation and machine learning). The MIT Press, Cambridge (1998)
Weber, C., Elshaw, M., Wermter, S., Triesch, J., Willmot, C.: Reinforcement learning embedded in brains and robots. In: Reinforcement learning: Theory and applications, pp. 119–142. InTech Education and Publishing (2008)
Weber, C., Triesch, J.: Goal-directed feature learning. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN, pp. 3355–3362. IEEE Press, Piscataway (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Navarro, N., Weber, C., Wermter, S. (2011). Real-World Reinforcement Learning for Autonomous Humanoid Robot Charging in a Home Environment. In: Groß, R., Alboul, L., Melhuish, C., Witkowski, M., Prescott, T.J., Penders, J. (eds) Towards Autonomous Robotic Systems. TAROS 2011. Lecture Notes in Computer Science(), vol 6856. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23232-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-23232-9_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23231-2
Online ISBN: 978-3-642-23232-9
eBook Packages: Computer ScienceComputer Science (R0)