Skip to main content
SpringerLink
Log in

Social Optimality in Cooperative Multiagent Systems

  • Chapter
  • First Online:
Interactions in Multiagent Systems: Fairness, Social Optimality and Individual Rationality

  • 479 Accesses

Abstract

In this chapter, we turn to look at another important solution concept called social optimality which targets at maximizing the sum of all agents’ payoffs involved. A socially optimal outcome is desirable in that it is not only optimal from the system-level’s perspective but also Pareto optimal. To achieve socially optimal outcomes in cooperative environments, the major challenge is how each agent can coordinate effectively with others given limited information, since the behaviors of other agents coexisting in the system may significantly impede the coordination process among them. The coordination problem becomes more difficult when the environment is uncertain (or stochastic) and each agent can only interact with its local partners if we consider a topology-based interaction environment [1, 6].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In the current implementation, we only take each agent’s payoff during the second-half period into consideration in order to get a more accurate evaluation of the actual performance of the period.

  2. 2.

    In the current implementation, we only take each agent’s payoff during the second-half period into consideration in order to get a more accurate evaluation of the actual performance of the period.

  3. 3.

    Notice that socially optimal outcomes here correspond to Pareto-optimal Nash equilibria in the transformed game \(G^{{\prime}}\) under the social learning update scheme.

  4. 4.

    Note that these two examples are the most commonly adopted test beds in previous work [4, 5, 20, 22].

  5. 5.

    Note that the minimum unit of the agents’ utilities is 1 here, since the utility function is defined in integers.

  6. 6.

    Note that the gray nodes 3 and 15 do not belong to the negotiation tree itself and for explanation only.

  7. 7.

    Note that each agent only knows its own best possible utility over each node in the negotiation tree. Here we show both agents’ information in the same negotiation tree for illustration purpose only.

  8. 8.

    Recall that we assume that the agent’s utilities are integers only and the utility’s minimum unit is 1. Since the agents are altruistic-individually rational, and also u 1(A 0(1)) = u i (A 13(1)), agent 1 will ask for a payment of p(1) = 1 to have the incentive to propose the allocation A 13.

  9. 9.

    Note that this is based on the assumption that the agents are altruistic-individually rational. This assumption is important to prevent that the socially optimal allocation may be discarded during negotiation. For example, consider a deal (A t , A t+1) in which A t+1 is the socially rational allocation, and \(u_{1}(A_{t}(1)) = 10,u_{2}(A_{t}(2)) = 6,u_{1}(A_{t+1}(1)) = 15\), and \(u_{2}(A_{t+1}(2)) = 2\). Without this assumption, agent 1 may propose the deal (A t , A t+1) with p(1) = 3, and accordingly, agent 2 will reject this offer since its utility is decreased.

References

  1. Hoen PJ, Tuyls Kl, Panait L, Luke S, Poutre JAL (2005) An overview of cooperative and competitive multiagent learning. In: Proceedings of first international workshop on learning and adaption in multi-agent systems, Utrecht, pp 1–46

    Google Scholar 

  2. Hao JY, Leung HF (2013) Reinforcement social learning of coordination in cooperative multi-agent systems(extended abstract). In: Proceedings of AAMAS’13, Saint Paul, pp 1321–1322

    Google Scholar 

  3. Hao JY, Leung HF (2013) The dynamics of reinforcement social learning in cooperative multiagent systems. In: Proceedings of IJCAI 13, Beijing, pp 184–190

    Google Scholar 

  4. Matlock M, Sen S (2007) Effective tag mechanisms for evolving coordination. In: Proceedings of AAMAS’07, Toronto, p 251

    Google Scholar 

  5. Hao JY, Leung HF (2011) Learning to achieve social rationality using tag mechanism in repeated interactions. In: Proceedings of ICTAI’11, Washington, DC, pp 148–155

    Google Scholar 

  6. Panait L, Luke S (2005) Cooperative multi-agent learning: the state of the art. Auton Agents Multi-Agent Syst 11(3):387–434

    Article  Google Scholar 

  7. Fulda N, Ventura D (2007) Predicting and preventing coordination problems in cooperative learning systems. In: Proceedings of IJCAI’07, Hyderabad

    Google Scholar 

  8. Matignon L, Laurent GJ, Le For-Piat N (2012) Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems. Knowl Eng Rev 27:1–31

    Article  Google Scholar 

  9. Claus C, Boutilier C (1998) The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of AAAI’98, Madison, pp 746–752

    Google Scholar 

  10. Lauer M, Rienmiller M (2000) An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In: Proceedings of ICML’00, Stanford, pp 535–542

    Google Scholar 

  11. Kapetanakis S, Kudenko D (2002) Reinforcement learning of coordination in cooperative multiagent systems. In: Proceedings of AAAI’02, Edmonton, pp 326–331

    Google Scholar 

  12. Matignon L, Laurent GJ, Le For-Piat N (2008) A study of FMQ heuristic in cooperative multi-agent games. In: AAMAS’08 workshop: MSDM, Estoril, pp 77–91

    Google Scholar 

  13. Panait L, Sullivan K, Luke S (2006) Lenient learners in cooperative multiagent systems. In: Proceedings of AAMAS’06, Utrecht, pp 801–803

    Google Scholar 

  14. Wang X, Sandholm T (2002) Reinforcement learning to play an optimal nash equilibrium in team Markov games. In: Proceedings of NIPS’02, Vancouver, pp 1571–1578

    Google Scholar 

  15. Brafman RI, Tennenholtz M (2004) Efficient learning equilibrium. Artif Intell 159:27–47

    Article  MathSciNet  MATH  Google Scholar 

  16. Watkins CJCH, Dayan PD (1992) Q-learning. Mach Learn 8:279–292

    MATH  Google Scholar 

  17. Melo FS, Veloso M (2009) Learning of coordination: exploiting sparse interactions in multiagent systems. In: Proceedings of AAMAS’09, Budapest, pp 7730–780

    Google Scholar 

  18. Sen S, Airiau S (2007) Emergence of norms through social learning. In: Proceedings of IJCAI’07, Hyderabad, pp 1507–1512

    Google Scholar 

  19. Fudenberg D, Levine DK (1998) The theory of learning in games. MIT, Cambridge

    MATH  Google Scholar 

  20. Hales D, Edmonds B (2003) Evolving social rationality for mas using “tags”. In: Proceedings of AAMAS’03. ACM, New York, pp 497–503

    Google Scholar 

  21. Sen S, Arora N, Roychowdhury S (1998) Using limited information to enhance group stability. Int J Hum-Comput Stud 48:69–82

    Article  Google Scholar 

  22. Matlock M, Sen S (2009) Effective tag mechanisms for evolving coperation. In: Proceedings of AAMAS’09, Budapest, pp 489–496

    Google Scholar 

  23. Hogg LM, Jennings NR (1997) Socially rational agents. In: Proceedings of AAAI fall symposium on socially intelligent agents, Providence, pp 61–63

    Google Scholar 

  24. Hogg LMJ, Jennings NR (2001) Socially intelligent reasoning for autonomous agents. IEEE Trans SMC Part A Syst Hum 31:381–393

    Article  Google Scholar 

  25. Chao I, Ardaiz O, Sanguesa R (2008) Tag mechanisms evaluated for coordination in open multi-agent systems. In: Proceedings of 8th international workshop on engineering societies in the agents world, Athens, pp 254–269

    Google Scholar 

  26. Chevaleyre Y, Dunne PE et al (2006) Issues in multiagent resource allocation. Informatica 30:3–31

    MATH  Google Scholar 

  27. Chevaleyre Y, Endriss U, Maudet N (2006) Tractable negotiation in tree-structured domains. In: Proceedings of AAMAS’06, Hakodate, pp 362–369

    Google Scholar 

  28. Endriss U, Maudet N (2005) On the communication complexity of multilateral trading: extended report. Auton Agents Multi-Agent Syst 11:91–107

    Google Scholar 

  29. Saha S, Sen S (2007) An efficient protocol for negotiation over multiple indivisible resources. In: Proceedings of IJCAI’07, Hyderabad, pp 1494–1499

    Google Scholar 

  30. Maly K, Overstreet C, Qiu X, Tang D (1988) Dynamic bandwidth allocation in a network. In: Proceedings of the ACM symposium on communications architectures and protocols, Stanford

    Book  Google Scholar 

  31. Gomoluch J, Schroeder M (2003) Market-based resource allocation for grid computing: a model and simulation. In: Proceedings of MGC’03, Rio de Janeiro, pp 211–218

    Google Scholar 

  32. Endriss U, Maudet N, Sadri F, Toni F (2003) On optimal outcomes of negotiation over resources. In: Proceedings of AAMAS’03, Melbourne

    Google Scholar 

  33. Chevaleyre Y, Endriss U, Maudet N (2010) Simple negotiation schemes for agents with simple preferences: sufficiency, necessity and maximality. Auton Agents Multi-Agent Syst 20(2):234–259

    Article  Google Scholar 

  34. Rosenschein JS, Zlotkin G (1994) Rules of encounter. MIT, Cambridge

    MATH  Google Scholar 

  35. Brams SJ, Taylor AD (2000) The win-win solution: guaranteeing fair shares to everybody. W.W.Norton and Company, New York

    Google Scholar 

  36. Endriss U, Maudet N (2003) Welfare engineering in multiagent systems. In: Engineering societies in the agents world IV. Springer, Berlin, pp 93–106

    Google Scholar 

  37. Arrow KJ, Sen AK, Suzumura K (2002) Handbook of social choice and welfare. North-Holland, Amsterdam

    MATH  Google Scholar 

  38. Brams SJ, Taylor AD (1996) Fair division: from cake-cutting to dispute resolution. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  39. Rubinstein A (1982) Perfect equilibrium in a bargaining model. Econometrica 50(1):97–110

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Software, Tianjin University, Tianjin, China

    Jianye Hao

  2. Department of Computer Sci. and Eng., The Chinese University of Hong Kong, Hong Kong, China

    Ho-fung Leung

Authors
  1. Jianye Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ho-fung Leung
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Higher Education Press, Beijing and Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Hao, J., Leung, Hf. (2016). Social Optimality in Cooperative Multiagent Systems. In: Interactions in Multiagent Systems: Fairness, Social Optimality and Individual Rationality. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-49470-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-49470-7_4

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-49468-4

  • Online ISBN: 978-3-662-49470-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation