Abstract
This paper proposes a new Reinforcement Learning (RL) algorithm for formation of agents in regular geometric forms. Due to curse of dimensionality problem, applying RL algorithms in formation problems cannot present suitable performance. Moreover, since the state space in formation problem is large, this leads to long learning time. Here, a multi-agent fuzzy reinforcement learning algorithm is presented that is an extension of fuzzy actor-critic reinforcement learning in a multi-agent environment. The final action for each agent is generated by a zero order T-S fuzzy system. In conventional fuzzy actor-critic RL, there are several candidate actions for consequence of each fuzzy rule and aim of learning is finding the best action among these discrete candidate actions. Here, using the proposed linear interpolation, a continuous action selection for determining the best action for each fuzzy rule is presented. The simulation results show the proposed method can improve the learning speed and action quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bik, J.J.C.M., Visser, P.N.A.M., Jennrich, O.: LISA satellite formation control. Adv. Space Res. 40, 25–34 (2007)
Breivik, M.: Topics in guided motion control of marine vehicles. Ph.D. thesis, Norwegian University of Science and Technology (2010)
Chen, G., Cao, W., Chen, X., Wu, M.: Multi-agent q-learning with joint state value approximation. In: Proceedings of the 30th Chinese Control Conference, p. 48784882 (2011)
Chen, X., Chen, G., Cao, W., Wu, M.: Cooperative learning with joint state value approximation for multi-agent systems. J. Control Theory Appl. 11, 149–155 (2013)
Derhami, V., Majd, V.J., Ahmadabadi, M.N.: Fuzzy sarsa learning and the proof of existence of Its stationary points. Asian J. Control 10, 535–549 (2008)
Dierks, T., Jagannathan, S.: Neural network output feedback control of robot formations. IEEE Trans. Syst., Man, Cybern. Part B 40, pp. 383-399 (2010)
Feddema, J., Lewis, C., Schoenwald, D.: Decentralized control of cooperative robotic vehicles: theory and application. IEEE Trans. Robot. Autom 18(5), 852864 (2002)
Franco, F.E., Waissman, V.J., Garca, L.J.: Learning the filling policy of a biodegradation process by fuzzy actorcritic learning methodology. Adv. Artif. Intell. 5317, 243–253 (2008)
Izzo, D., Pettazzi, L.: Autonomous and distributed motion planning for satellite swarm. J. Guid. Control Dyn. 30, 449459 (2005)
Macdonald, E.A.: Multi-robot assignment and formation control. M.Sc. thesis, Georgia Institute of Technology (2011)
Panait, L., Luke, S.: Cooperative multi-agent learning: the state of the art. Auton. Agents Multi-agent Syst. 11, 387–434 (2005)
Sanz, Y., de Lope, J., Martn, J.A.H.: Applying reinforcement learing to multi-robot team coordination. In: Corchado, E., Abraham, A., Pedrycz, W. (eds.) HAIS 2008. LNCS, vol. 5271, pp. 625632. Springer, Heidelberg (2008)
Zuo, G., Han, J., Han, G.: Multi-robot formation control using reinforcement learning, advances in swarm Intelligence. vol. 6145, pp. 667–674. Springer, Berlin (2010)
Acknowledgments
This research was financially supported by the Center of Excellence for Robust and Intelligence Systems (CERIS) of Yazd University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Derhami, V., Momeni, Y. (2016). Applying Reinforcement Learning in Formation Control of Agents. In: Novais, P., Camacho, D., Analide, C., El Fallah Seghrouchni, A., Badica, C. (eds) Intelligent Distributed Computing IX. Studies in Computational Intelligence, vol 616. Springer, Cham. https://doi.org/10.1007/978-3-319-25017-5_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-25017-5_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25015-1
Online ISBN: 978-3-319-25017-5
eBook Packages: EngineeringEngineering (R0)