Abstract
Towards a compact and elaboration-tolerant first-order representation of Markov games, we introduce relational Markov games, which combine standard Markov games with first-order action descriptions in a stochastic variant of the situation calculus. We focus on the zero-sum two-agent case, where we have two agents with diametrically opposed goals. We also present a symbolic value iteration algorithm for computing Nash policy pairs in this framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Boutilier, C.: Sequential optimality and coordination in multiagent systems. In: Proceedings IJCAI 1999, pp. 478–485 (1999)
Boutilier, C., Reiter, R., Price, B.: Symbolic dynamic programming for first-order MDPs. In: Proceedings IJCAI 2001, pp. 690–700 (2001)
Boutilier, C., Reiter, R., Soutchanski, M., Thrun, S.: Decision-theoretic, high-level agent programming in the situation calculus. In: Proceedings AAAI 2000, pp. 355–362 (2000)
Bowling, M., Veloso, M.: Multiagent learning using a variable learning rate. Artif. Intell. 136(2), 215–250 (2002)
Finzi, A., Lukasiewicz, T.: Game-theoretic agent programming in Golog. In: Proceedings ECAI 2004 (2004) (to appear)
Gardiol, N.H., Kaelbling, L.P.: Envelope-based planning in relational MDPs. In: Proceedings NIPS 2003 (2003)
Guestrin, C., Koller, D., Gearhart, C., Kanodia, N.: Generalizing plans to new environments in relational MDPs. In: Proceedings IJCAI 2003, pp. 1003–1010 (2003)
Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99–134 (1998)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings ICML 1994, pp. 157–163 (1994)
McCarthy, J., Hayes, P.J.: Some philosophical problems from the standpoint of artificial intelligence. Machine Intelligence 4, 463–502 (1969)
Owen, G.: Game Theory, 2nd edn. Academic Press, London (1982)
Poole, D.: The independent choice logic for modelling multiple agents under uncertainty. Artif. Intell. 94, 7–56 (1997)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, Chichester (1994)
Reiter, R.: The frame problem in the situation calculus: A simple solution (sometimes) and a completeness result for goal regression. In: Artificial Intelligence and Mathematical Theory of Computation: Papers in Honor of John McCarthy, pp. 359–380. Academic Press, London (1991)
Reiter, R.: Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems. MIT Press, Cambridge (2001)
van der Wal, J.: Stochastic Dynamic Programming. Mathematical Centre Tracts, vol. 139. Morgan Kaufmann, San Francisco (1981)
von Neumann, J., Morgenstern, O.: The Theory of Games and Economic Behavior. Princeton University Press, Princeton (1947)
Yoon, S.W., Fern, A., Givan, B.: Inductive policy selection for first-order MDPs. In: Proceedings UAI 2002, pp. 569–576 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Finzi, A., Lukasiewicz, T. (2004). Relational Markov Games. In: Alferes, J.J., Leite, J. (eds) Logics in Artificial Intelligence. JELIA 2004. Lecture Notes in Computer Science(), vol 3229. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30227-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-30227-8_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23242-1
Online ISBN: 978-3-540-30227-8
eBook Packages: Springer Book Archive