Skip to main content
SpringerLink
Log in

Zero-Sum Stochastic Games

  • Reference work entry
  • First Online:
Handbook of Dynamic Game Theory

Abstract

In this chapter, we describe a major part of the theory of zero-sum discrete-time stochastic games. We review all basic streams of research in this area, in the context of the existence of value and uniform value, algorithms, vector payoffs, incomplete information, and imperfect state observation. Also some models related to continuous-time games, e.g., games with short-stage duration, semi-Markov games, are mentioned. Moreover, a number of applications of stochastic games are pointed out. The provided reference list reveals a tremendous progress made in the field of zero-sum stochastic games since the seminal work of Shapley (Proc Nat Acad Sci USA 39:1095–1100, 1953).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 899.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aliprantis C, Border K (2006) Infinite dimensional analysis: a Hitchhiker’s guide. Springer, New York

    Google Scholar 

  • Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Meth Oper Res 62:375–386

    Google Scholar 

  • Altman E, Feinberg EA, Shwartz A (2000) Weighted discounted stochastic games with perfect information. Annals of the International Society of Dynamic Games, vol 5. Birkhäuser, Boston, pp 303–324

    Google Scholar 

  • Altman E, Gaitsgory VA (1995) A hybrid (differential-stochastic) zero-sum game with fast stochastic part. Annals of the International Society of Dynamic Games, vol 3. Birkhäuser, Boston, pp 47–59

    Google Scholar 

  • Altman E, Hordijk A (1995) Zero-sum Markov games and worst-case optimal control of queueing systems. Queueing Syst Theory Appl 21:415–447

    Google Scholar 

  • Altman E, Hordijk A, Spieksma FM (1997) Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards. Math Oper Res 22:588–618

    Google Scholar 

  • Arapostathis A, Borkar VS, Fernández-Gaucherand, Gosh MK, Markus SI (1993) Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J Control Optim 31:282–344

    Google Scholar 

  • Avrachenkov K, Cottatellucci L, Maggi L (2012) Algorithms for uniform optimal strategies in two-player zero-sum stochastic games with perfect information. Oper Res Lett 40:56–60

    Google Scholar 

  • Basu S (1999) New results on quantifier elimination over real-closed fields and applications to constraint databases. J ACM 46:537–555

    Google Scholar 

  • Basu S, Pollack R, Roy MF (2003) Algorithms in real algebraic geometry. Springer, New York

    Google Scholar 

  • Basu A, Stettner Ł (2015) Finite- and infinite-horizon Shapley games with non-symmetric partial observation. SIAM J Control Optim 53:3584–3619

    Google Scholar 

  • Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic, New York

    Google Scholar 

  • Berge C (1963) Topological spaces. MacMillan, New York

    Google Scholar 

  • Bertsekas DP, Shreve SE (1996) Stochastic Optimal Control: the Discrete-Time Case. Athena Scientic, Belmont

    Google Scholar 

  • Bewley T, Kohlberg E (1976a) The asymptotic theory of stochastic games. Math Oper Res 1:197–208

    Google Scholar 

  • Bewley T, Kohlberg E (1976b) The asymptotic solution of a recursion equation occurring in stochastic games. Math Oper Res 1:321–336

    Google Scholar 

  • Bewley T, Kohlberg E (1978) On stochastic games with stationary optimal strategies. Math Oper Res 3:104–125

    Google Scholar 

  • Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, Cambridge

    Google Scholar 

  • Billingsley P (1968) Convergence of probability measures. Wiley, New York

    Google Scholar 

  • Blackwell DA, Girshick MA (1954) Theory of games and statistical decisions. Wiley and Sons, New York

    Google Scholar 

  • Blackwell D (1956) An analog of the minmax theorem for vector payoffs. Pac J Math 6:1–8

    Google Scholar 

  • Blackwell D (1962) Discrete dynamic programming. Ann Math Statist 33:719–726

    Google Scholar 

  • Blackwell D (1965) Discounted dynamic programming. Ann Math Statist 36: 226–235

    Google Scholar 

  • Blackwell D (1969) Infinite Gδ-games with imperfect information. Zastosowania Matematyki (Appl Math) 10:99–101

    Google Scholar 

  • Blackwell D (1989) Operator solution of infinite Gδ-games of imperfect information. In: Anderson TW et al (eds) Probability, Statistics, and Mathematics: Papers in Honor of Samuel Karlin. Academic, New York, pp 83–87

    Google Scholar 

  • Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39:159–163

    Google Scholar 

  • Bolte J, Gaubert S, Vigeral G (2015) Definable zero-sum stochastic games. Math Oper Res 40: 80–104

    Google Scholar 

  • Boros E, Elbassioni K, Gurvich V, Makino K (2013) On canonical forms for zero-sum stochastic mean payoff games. Dyn Games Appl 3:128–161

    Google Scholar 

  • Boros E, Elbassioni K, Gurvich V, Makino K (2016) A potential reduction algorithm for two-person zero-sum mean payoff stochastic games. Dyn Games Appl doi:10.1007/s13235-016-0199-x

    Google Scholar 

  • Breton M (1991) Algorithms for stochastic games. In: Stochastic games and related topics. Shapley honor volume. Kluwer, Dordrecht, pp 45–58

    Google Scholar 

  • Brown LD, Purves R (1973) Measurable selections of extrema. Ann Stat 1:902–912

    Google Scholar 

  • Cardaliaguet P, Laraki R, Sorin S (2012) A continuous time approach for the asymptotic value in two-person zero-sum repeated games. SIAM J Control Optim 50:1573–1596

    Google Scholar 

  • Cardaliaguet P, Rainer C, Rosenberg D, Vieille N (2016) Markov games with frequent actions and incomplete information-The limit case. Math Oper Res 41:49–71

    Google Scholar 

  • Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, Cambridge

    Google Scholar 

  • Cesa-Bianchi N, Lugosi G, Stoltz G (2006) Regret minimization under partial monitoring. Math Oper Res 31:562–580

    Google Scholar 

  • Charnes A, Schroeder R (1967) On some tactical antisubmarine games. Naval Res Logistics Quarterly 14:291–311

    Google Scholar 

  • Chatterjee K, Majumdar R, Henzinger TA (2008) Stochastic limit-average games are in EXPTIME. Int J Game Theory 37:219–234

    Google Scholar 

  • Condon A (1992) The complexity of stochastic games. Inf Comput 96:203–224

    Google Scholar 

  • Coulomb JM (1992) Repeated games with absorbing states and no signals. Int J Game Theory 21:161–174

    Google Scholar 

  • Coulomb JM (1999) Generalized big-match. Math Oper Res 24:795–816

    Google Scholar 

  • Coulomb JM (2001) Absorbing games with a signalling structure. Math Oper Res 26:286–303

    Google Scholar 

  • Coulomb JM (2003a) Absorbing games with a signalling structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 335–355

    Google Scholar 

  • Coulomb JM (2003b) Games with a recursive structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 427–442

    Google Scholar 

  • Coulomb JM (2003c) Stochastic games with imperfect monitoring. Int J Game Theory (2003) 32:73–96

    Google Scholar 

  • Couwenbergh HAM (1980) Stochastic games with metric state spaces. Int J Game Theory 9:25–36

    Google Scholar 

  • de Alfaro L, Henzinger TA, Kupferman O (2007) Concurrent reachability games. Theoret Comp Sci 386:188–217

    Google Scholar 

  • Dubins LE (1957) A discrete invasion game. In: Dresher M et al (eds) Contributions to the theory of games III. Annals of Mathematics Studies, vol 39. Princeton University Press, Princeton, pp 231–255

    Google Scholar 

  • Dubins LE, Maitra A, Purves R, Sudderth W (1989) Measurable, nonleavable gambling problems. Israel J Math 67:257–271

    Google Scholar 

  • Dubins LE, Savage LJ (2014) Inequalities for stochastic processes. Dover, New York

    Google Scholar 

  • Everett H (1957) Recursive games. In: Dresher M et al (eds) Contributions to the theory of games III. Annals of Mathematics Studies, vol 39. Princeton University Press, Princeton, pp 47–78

    Google Scholar 

  • Fan K (1953) Minmax theorems. Proc Nat Acad Sci USA 39:42–47

    Google Scholar 

  • Feinberg EA (1994) Constrained semi-Markov decision processes with average rewards. Math Methods Oper Res 39:257–288

    Google Scholar 

  • Feinberg EA, Lewis ME (2005) Optimality of four-threshold policies in inventory systems with customer returns and borrowing/storage options. Probab Eng Inf Sci 19:45–71

    Google Scholar 

  • Filar JA (1981) Ordered field property for stochastic games when the player who controls transitions changes from state to state. J Optim Theory Appl 34:503–513

    Google Scholar 

  • Filar JA (1985). Player aggregation in the travelling inspector model. IEEE Trans Autom Control 30:723–729

    Google Scholar 

  • Filar JA, Schultz TA, Thuijsman F, Vrieze OJ (1991) Nonlinear programming and stationary equilibria of stochastic games. Math Program Ser A 50:227–237

    Google Scholar 

  • Filar JA, Tolwinski B (1991) On the algorithm of Pollatschek and Avi-Itzhak. In: Stochastic games and related topics. Shapley honor volume. Kluwer, Dordrecht, pp 59–70

    Google Scholar 

  • Filar JA, Vrieze, OJ (1992) Weighted reward criteria in competitive Markov decision processes. Z Oper Res 36:343–358

    Google Scholar 

  • Filar JA, Vrieze K (1997) Competitive Markov decision processes. Springer, New York

    Google Scholar 

  • Flesch J, Thuijsman F, Vrieze OJ (1999) Average-discounted equilibria in stochastic games. European J Oper Res 112:187–195

    Google Scholar 

  • Fristedt B, Lapic S, Sudderth WD (1995) The big match on the integers. Annals of the international society of dynamic games, vol 3. Birkhä user, Boston, pp 95–107

    Google Scholar 

  • Gale D, Steward EM (1953) Infinite games with perfect information. In: Kuhn H, Tucker AW (eds) Contributions to the theory of games II. Annals of mathematics studies, vol 28. Princeton University Press, Princeton, pp 241–266

    Google Scholar 

  • Gensbittel F (2016) Continuous-time limit of dynamic games with incomplete information and a more informed player. Int J Game Theory 45:321–352

    Google Scholar 

  • Gensbittel F, Oliu-Barton M, Venel X (2014) Existence of the uniform value in repeated games with a more informed controller. J Dyn Games 1:411–445

    Google Scholar 

  • Gensbittel F, Renault J (2015) The value of Markov chain games with lack of information on both sides. Math Oper Res 40:820–841

    Google Scholar 

  • Gillette D (1957) Stochastic games with zero stop probabilities.In: Dresher M et al (eds) Contributions to the theory of games III. Annals of mathematics studies, vol 39. Princeton University Press, Princeton, pp 179–187

    Google Scholar 

  • Ghosh MK, Bagchi A (1998) Stochastic games with average payoff criterion. Appl Math Optim 38:283–301

    Google Scholar 

  • Gimbert H, Renault J, Sorin S, Venel X, Zielonka W (2016) On values of repeated games with signals. Ann Appl Probab 26:402–424

    Google Scholar 

  • González-Trejo JI, Hernández-Lerma O, Hoyos-Reyes LF (2003) Minmax control of discrete-time stochastic systems. SIAM J Control Optim 41:1626–1659

    Google Scholar 

  • Guo X, Hernández-Lerma O (2003) Zero-sum games for continuous-time Markov chains with unbounded transitions and average payoff rates. J Appl Probab 40:327–345

    Google Scholar 

  • Guo X, Hernńdez-Lerma O (2005) Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs. J Appl Probab 42:303–320

    Google Scholar 

  • Hall P, Heyde C (1980) Martingale limit theory and its applications. Academic, New York

    Google Scholar 

  • Hansen LP, Sargent TJ (2008) Robustness. Princeton University Press, Princeton

    Google Scholar 

  • Haurie A, Krawczyk JB, Zaccour G (2012) Games and dynamic games. World Scientific, Singapore

    Google Scholar 

  • Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer-Verlag, New York

    Google Scholar 

  • Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New York

    Google Scholar 

  • Himmelberg CJ (1975) Measurable relations. Fundam Math 87:53–72

    Google Scholar 

  • Himmelberg CJ, Van Vleck FS (1975) Multifunctions with values in a space of probability measures. J Math Anal Appl 50:108–112

    Google Scholar 

  • Hoffman AJ, Karp RM (1966) On non-terminating stochastic games. Management Sci 12:359–370

    Google Scholar 

  • Hordijk A, Kallenberg LCM (1981) Linear programming and Markov games I, II. In: Moeschlin O, Pallaschke D (eds) Game theory and mathematical economics, North-Holland, Amsterdam, pp 291–320

    Google Scholar 

  • Iyengar GN (2005) Robust dynamic programming. Math Oper Res 30:257–280

    Google Scholar 

  • Jaśkiewicz A (2002) Zero-sum semi-Markov games. SIAM J Control Optim 41:723–739

    Google Scholar 

  • Jaśkiewicz A (2004) On the equivalence of two expected average cost criteria for semi-Markov control processes. Math Oper Res 29:326–338

    Google Scholar 

  • Jaśkiewicz A (2007) Average optimality for risk-sensitive control with general state space. Ann Appl Probab 17: 654–675

    Google Scholar 

  • Jaśkiewicz A (2009) Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities. J Optim Theory Appl 141:321–347

    Google Scholar 

  • Jaśkiewicz A (2010) On a continuous solution to the Bellman-Poisson equation in stochastic games. J Optim Theory Appl 145:451–458

    Google Scholar 

  • Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45:773–789

    Google Scholar 

  • Jaśkiewicz A, Nowak AS (2011) Stochastic games with unbounded payoffs: Applications to robust control in economics. Dyn Games Appl 1: 253–279

    Google Scholar 

  • Jaśkiewicz A, Nowak AS (2014) Robust Markov control process. J Math Anal Appl 420:1337–1353

    Google Scholar 

  • Jaśkiewicz A, Nowak AS (2018) Non-zero-sum stochastic games. In: Başar T, Zaccour G (eds) Handbook of dynamic game theory. Birkhäuser, Basel

    Google Scholar 

  • Kalathil, D, Borkar VS, Jain R (2016) Approachability in Stackelberg stochastic games with vector costs. Dyn Games Appl. doi:10.1007/s13235–016-0198-y

    Google Scholar 

  • Kartashov NV(1996) Strong stable Markov chains. VSP, Utrecht, The Netherlands

    Google Scholar 

  • Kehagias A, Mitschke D, Praat P (2013) Cops and invisible robbers: The cost of drunkenness. Theoret Comp Sci 481:100–120

    Google Scholar 

  • Kertz RP, Nachman D (1979) Persistently optimal plans for nonstationary dynamic programming: The topology of weak convergence case. Ann Probab 7:811–826

    Google Scholar 

  • Klein E, Thompson AC (1984) Theory of correspondences. Wiley, New York

    Google Scholar 

  • Kohlberg E (1974) Repeated games with absorbing states. Ann Statist 2:724–738

    Google Scholar 

  • Krass D, Filar JA, Sinha S (1992) A weighted Markov decision process. Oper Res 40:1180–1187

    Google Scholar 

  • Krausz A, Rieder U (1997) Markov games with incomplete information. Math Meth Oper Res 46:263–279

    Google Scholar 

  • Krishnamurthy N, Parthasarathy T (2011) Multistage (stochastic) games. Wiley encyclopedia of operations research and management science. Wiley online library. doi:10.1002/9780470400531.eorms0551

    Google Scholar 

  • Kumar PR, Shiau TH (1981) Existence of value and randomized strategies in zero-sum discrete-time stochastic dynamic games. SIAM J Control Optim 19:617–634

    Google Scholar 

  • Kuratowski K, Ryll-Nardzewski C (1965) A general theorem on selectors. Bull Polish Acad Sci (Ser Math) 13:397–403

    Google Scholar 

  • Küenle HU (1986) Stochastische Spiele und Entscheidungsmodelle. BG Teubner, Leipzig

    Google Scholar 

  • Küenle HU (2007) On Markov games with average reward criterion and weakly continuous transition probabilities. SIAM J Control Optim 45:2156–2168

    Google Scholar 

  • Laraki R (2010) Explicit formulas for repeated games with absorbing states. Int J Game Theory 39:53–69

    Google Scholar 

  • Laraki R, Sorin S (2015) Advances in zero-sum dynamic games. In: Young HP, Zamir S (eds) Handbook of game theory with economic applications, vol 4. North Holland, pp 27–93

    Google Scholar 

  • Lehrer E, Solan E (2016) A general internal regret-free strategy. Dyn Games Appl 6:112–138

    Google Scholar 

  • Lehrer E, Sorin S (1992) A uniform Tauberian theorem in dynamic programming. Math Oper Res 17:303–307

    Google Scholar 

  • Levy Y (2012) Stochastic games with information lag. Games Econ Behavior 74:243–256

    Google Scholar 

  • Levy Y (2013) Continuous-time stochastic games of fixed duration. Dyn Games Appl 3:279–312

    Google Scholar 

  • Li X, Venel X (2016) Recursive games: uniform value, Tauberian theorem and the Mertens conjecture “\( Maxmin= \lim v_n=\lim v_\lambda \)”. Int J Game Theory 45:155–189

    Google Scholar 

  • Maccheroni F, Marinacci M, Rustichini A (2006) Dynamic variational preferences. J Econ Theory 128:4–44

    Google Scholar 

  • Maitra A, Parthasarathy T (1970) On stochastic games. J Optim Theory Appl 5:289–300

    Google Scholar 

  • Maitra A, Parthasarathy T (1971) On stochastic games II. J Optim Theory Appl 8:154–160

    Google Scholar 

  • Maitra A, Sudderth W (1992) An operator solution for stochastic games. Israel J Math 78:33–49

    Google Scholar 

  • Maitra A, Sudderth W (1993a) Borel stochastic games with limsup payoffs. Ann Probab 21:861–885

    Google Scholar 

  • Maitra A, Sudderth W (1993b) Finitely additive and measurable stochastic games. Int J Game Theory 22:201–223

    Google Scholar 

  • Maitra A, Sudderth W (1996) Discrete gambling and stochastic games. Springer, New York

    Google Scholar 

  • Maitra A, Sudderth W (1998) Finitely additive stochastic games with Borel measurable payoffs. Int J Game Theory 27:257–267

    Google Scholar 

  • Maitra A, Sudderth W (2003a) Stochastic games with limsup payoff. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 357–366

    Google Scholar 

  • Maitra A, Sudderth W (2003b) Stochastic games with Borel payoffs. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 367–373

    Google Scholar 

  • Martin D (1975) Borel determinacy. Ann Math 102:363–371

    Google Scholar 

  • Martin D (1985) A purely inductive proof of Borel determinacy. In: Nerode A, Shore RA (eds) Recursion theory. Proceedings of symposia in pure mathematics, vol 42. American Mathematical Society, Providence, pp 303–308

    Google Scholar 

  • Martin D (1998) The determinacy of Blackwell games. J Symb Logic 63:1565–1581

    Google Scholar 

  • Mertens JF (1982) Repeated games: an overview of the zero-sum case. In: Hildenbrand W (ed) Advances in economic theory. Cambridge Univ Press, Cambridge, pp 175–182

    Google Scholar 

  • Mertens JF (1987) Repeated games. In: Proceedings of the international congress of mathematicians, American mathematical society, Berkeley, California, pp 1528–1577

    Google Scholar 

  • Mertens JF (2002) Stochastic games. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. North Holland, pp 1809–1832

    Google Scholar 

  • Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–56

    Article  MathSciNet  MATH  Google Scholar 

  • Mertens JF, Neyman A (1982) Stochastic games have a value. Proc Natl Acad Sci USA 79: 2145–2146

    Article  MathSciNet  MATH  Google Scholar 

  • Mertens JF, Neyman A, Rosenberg D (2009) Absorbing games with compact action spaces. Math Oper Res 34:257–262

    Article  MathSciNet  MATH  Google Scholar 

  • Mertens JF, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, Cambridge, MA

    Book  MATH  Google Scholar 

  • Meyn SP, Tweedie RL (1994) Computable bounds for geometric convergence rates of Markov chains. Ann Appl Probab 4:981–1011

    Article  MathSciNet  MATH  Google Scholar 

  • Meyn SP, Tweedie RL (2009) Markov chains and stochastic stability. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Miao J (2014) Economic dynamics in discrete time. MIT Press, Cambridge

    MATH  Google Scholar 

  • Milman E (2002) The semi-algebraic theory of stochastic games. Math Oper Res 27:401–418

    Article  MathSciNet  MATH  Google Scholar 

  • Milman E (2006) Approachable sets of vector payoffs in stochastic games. Games Econ Behavior 56:135–147

    Article  MathSciNet  MATH  Google Scholar 

  • Milnor J, Shapley LS (1957) On games of survival. In: Dresher M et al (eds) Contributions to the theory of games III. Annals of mathematics studies, vol 39. Princeton University Press, Princeton, pp 15–45

    MATH  Google Scholar 

  • Mondal P, Sinha S, Neogy SK, Das AK (2016) On discounted ARAT semi-Markov games and its complementarity formulations. Int J Game Theory 45:567–583

    Article  MATH  Google Scholar 

  • Mycielski J (1992) Games with perfect information. In: Aumann RJ, Hart S (eds) Handbook of Game Theory with Economic Applications, vol 1. North Holland, pp 41–70

    Google Scholar 

  • Neyman A (2003a) Real algebraic tools in stochastic games. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht, pp 57–75

    Chapter  MATH  Google Scholar 

  • Neyman A (2003b) Stochastic games and nonexpansive maps. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht, pp 397–415

    Chapter  MATH  Google Scholar 

  • Neyman A (2013) Stochastic games with short-stage duration. Dyn Games Appl 3:236–278

    Article  MathSciNet  MATH  Google Scholar 

  • Neyman A, Sorin S (eds) (2003) Stochastic games and applications. Kluwer, Dordrecht

    MATH  Google Scholar 

  • Neveu J (1965) Mathematical foundations of the calculus of probability. Holden-Day, San Francisco

    MATH  Google Scholar 

  • Nowak AS (1985a) Universally measurable strategies in zero-sum stochastic games. Ann Probab 13: 269–287

    Article  MathSciNet  MATH  Google Scholar 

  • Nowak AS (1985b) Measurable selection theorems for minmax stochastic optimization problems, SIAM J Control Optim 23:466–476

    Article  MathSciNet  MATH  Google Scholar 

  • Nowak AS (1986) Semicontinuous nonstationary stochastic games. J Math Analysis Appl 117:84–99

    Article  MathSciNet  MATH  Google Scholar 

  • Nowak AS (1994) Zero-sum average payoff stochastic games wit general state space. Games Econ Behavior 7:221–232

    Article  MathSciNet  MATH  Google Scholar 

  • Nowak AS (2010) On measurable minmax selectors. J Math Anal Appl 366:385–388

    Article  MathSciNet  MATH  Google Scholar 

  • Nowak AS, Raghavan TES (1991) Positive stochastic games and a theorem of Ornstein. In: Stochastic games and related topics. Shapley honor volume. Kluwer, Dordrecht, pp 127–134

    Chapter  MATH  Google Scholar 

  • Oliu-Barton M (2014) The asymptotic value in finite stochastic games. Math Oper Res 39:712–721

    Article  MathSciNet  MATH  Google Scholar 

  • Ornstein D (1969) On the existence of stationary optimal strategies. Proc Am Math Soc 20:563–569

    Article  MathSciNet  MATH  Google Scholar 

  • Papadimitriou CH (1994) Computational complexity. Addison-Wesley, Reading

    MATH  Google Scholar 

  • Parthasarathy KR (1967) Probability measures on metric spaces. Academic, New York

    Book  MATH  Google Scholar 

  • Parthasarathy T, Raghavan TES (1981) An order field property for stochastic games when one player controls transition probabilities. J Optim Theory Appl 33:375–392

    Article  MathSciNet  MATH  Google Scholar 

  • Patek SD, Bertsekas DP (1999) Stochastic shortest path games. SIAM J Control Optim 37:804–824

    Article  MathSciNet  MATH  Google Scholar 

  • Perchet V (2011a) Approachability of convex sets in games with partial monitoring. J Optim Theory Appl 149:665–677

    Article  MathSciNet  MATH  Google Scholar 

  • Perchet V (2011b) Internal regret with partial monitoring calibration-based optimal algorithms. J Mach Learn Res 12:1893–1921

    MathSciNet  MATH  Google Scholar 

  • Pollatschek M, Avi-Itzhak B (1969) Algorithms for stochastic games with geometrical interpretation. Manag Sci 15:399–425

    Article  MathSciNet  MATH  Google Scholar 

  • Prikry K, Sudderth WD (2016) Measurability of the value of a parametrized game. Int J Game Theory 45:675–683

    Article  MathSciNet  MATH  Google Scholar 

  • Raghavan TES (2003) Finite-step algorithms for single-controller and perfect information stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 227–251

    Chapter  MATH  Google Scholar 

  • Raghavan TES, Ferguson TS, Parthasarathy T, Vrieze OJ, eds. (1991) Stochastic games and related topics: In honor of professor LS Shapley, Kluwer, Dordrecht

    Google Scholar 

  • Raghavan TES, Filar JA (1991) Algorithms for stochastic games: a survey. Z Oper Res (Math Meth Oper Res) 35: 437–472

    Article  MathSciNet  MATH  Google Scholar 

  • Raghavan TES, Syed Z (2003) A policy improvement type algorithm for solving zero-sum two-person stochastic games of perfect information. Math Program Ser A 95:513–532

    Article  MathSciNet  MATH  Google Scholar 

  • Renault J (2006) The value of Markov chain games with lack of information on one side. Math Oper Res 31:490–512

    Article  MathSciNet  MATH  Google Scholar 

  • Renault J (2012) The value of repeated games with an uninformed controller. Math Oper Res 37:154–179

    Article  MathSciNet  MATH  Google Scholar 

  • Renault J (2014) General limit value in dynamic programming. J Dyn Games 1:471–484

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenberg D (2000) Zero-sum absorbing games with incomplete information on one side: asymptotic analysis. SIAM J Control Optim 39:208–225

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenberg D, Sorin S (2001) An operator approach to zero-sum repeated games. Israel J Math 121:221–246

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenberg D, Solan E, Vieille N (2002) Blackwell optimality in Markov decision processes with partial observation. Ann Stat 30:1178–1193

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenberg D, Solan E, Vieille N (2003) The maxmin value of stochastic games with imperfect monitoring. Int J Game Theory 32:133–150

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenberg D, Solan E, Vieille N (2004) Stochastic games with a single controller and incomplete information. SIAM J Control Optim 43:86–110

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenberg D, Vieille N (2000) The maxmin of recursive games with incomplete information on one side. Math Oper Res 25:23–35

    Article  MathSciNet  MATH  Google Scholar 

  • Scarf HE, Shapley LS (1957) A discrete invasion game. In: Dresher M et al (eds) Contributions to the Theory of Games III. Annals of Mathematics Studies, vol 39, Princeton University Press, Princeton, pp 213–229

    Google Scholar 

  • Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Z Wahrscheinlichkeitstheorie Verw Geb 32:179–196

    Article  MathSciNet  MATH  Google Scholar 

  • Secchi P (1997) Stationary strategies for recursive games. Math Oper Res 22:494–512

    Article  MathSciNet  MATH  Google Scholar 

  • Secchi P (1998) On the existence of good stationary strategies for nonleavable stochastic games. Int J Game Theory 27:61–81

    Article  MathSciNet  MATH  Google Scholar 

  • Selten R (1975) Re-examination of the perfectness concept for equilibrium points in extensive games. Int J Game Theory 4:25–55

    Article  MATH  Google Scholar 

  • Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  • Shimkin N, Shwartz A (1993) Guaranteed performance regions for Markovian systems with competing decision makers. IEEE Trans Autom Control 38:84–95

    Article  MathSciNet  MATH  Google Scholar 

  • Shmaya E (2011) The determinacy of infinite games with eventual perfect monitoring. Proc Am Math Soc 139:3665–3678

    Article  MathSciNet  MATH  Google Scholar 

  • Solan E (2009) Stochastic games. In: Meyers RA (ed) Encyclopedia of complexity and systems science. Springer, New York, pp 8698–8708

    Chapter  Google Scholar 

  • Solan E, Vieille N (2002) Uniform value in recursive games. Ann Appl Probab 12:1185–1201

    Article  MathSciNet  MATH  Google Scholar 

  • Solan E, Vieille N (2010) Camputing uniformly optimal strategies in two-player stochastic games. Econ Theory 42:237–253

    Article  MATH  Google Scholar 

  • Solan E, Vieille N (2015) Stochastic games. Proc Nat Acad Sci USA 112:13743–13746

    Article  MathSciNet  MATH  Google Scholar 

  • Solan E, Ziliotto B (2016) Stochastic games with signals. Annals of the international society of dynamic game, vol 14. Birkhäuser, Boston, pp 77–94

    Google Scholar 

  • Sorin S (1984) Big match with lack of information on one side (Part 1). Int J Game Theory 13:201–255

    Article  MathSciNet  MATH  Google Scholar 

  • Sorin S (1985) Big match with lack of information on one side (Part 2). Int J Game Theory 14:173–204

    Article  MathSciNet  MATH  Google Scholar 

  • Sorin S (2002) A first course on zero-sum repeated games. Mathematiques et applications, vol 37. Springer, New York

    Google Scholar 

  • Sorin S (2003a) Stochastic games with incomplete information. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht, pp 375–395

    Chapter  MATH  Google Scholar 

  • Sorin S (2003b) The operator approach to zero-sum stochastic games. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht

    MATH  Google Scholar 

  • Sorin S (2004) Asymptotic properties of monotonic nonexpansive mappings. Discrete Event Dyn Syst 14:109–122

    Article  MathSciNet  MATH  Google Scholar 

  • Sorin S, Vigeral G (2015a) Existence of the limit value of two-person zero-sum discounted repeated games via comparison theorems. J Optim Theory Appl 157:564–576

    Article  MathSciNet  MATH  Google Scholar 

  • Sorin S, Vigeral G (2015b) Reversibility and oscillations in zero-sum discounted stochastic games. J Dyn Games 2:103–115

    Article  MathSciNet  MATH  Google Scholar 

  • Sorin S, Vigeral G (2016) Operator approach to values of stochastic games with varying stage duration. Int J Game Theory 45:389–410

    Article  MathSciNet  MATH  Google Scholar 

  • Sorin S, Zamir S (1991) “Big match” with lack on information on one side (Part 3). In: Shapley LS, Raghavan TES (eds) Stochastic games and related topics. Shapley Honor Volume. Kluwer, Dordrecht, pp 101–112

    Chapter  Google Scholar 

  • Spinat X (2002) A necessary and sufficient condition for approachability. Math Oper Res 27:31–44

    Article  MathSciNet  MATH  Google Scholar 

  • Stokey NL, Lucas RE, Prescott E (1989) Recursive methods in economic dynamics. Harvard University Press, Cambridge

    Google Scholar 

  • Strauch R (1966) Negative dynamic programming. Ann Math Stat 37:871–890

    Article  MathSciNet  MATH  Google Scholar 

  • Szczechla W, Connell SA, Filar JA, Vrieze OJ (1997) On the Puiseux series expansion of the limit discount equation of stochastic games. SIAM J Control Optim 35:860–875

    Article  MathSciNet  MATH  Google Scholar 

  • Tarski A (1951) A decision method for elementary algebra and geometry. University of California Press, Berkeley

    MATH  Google Scholar 

  • Van der Wal I (1978) Discounted Markov games: Generalized policy iteration method. J Optim Theory Appl 25:125–138

    Article  MathSciNet  MATH  Google Scholar 

  • Vega-Amaya O (2003) Zero-sum average semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J Control Optim 42:1876–1894

    Article  MathSciNet  MATH  Google Scholar 

  • Vega-Amaya O, Luque-Vásquez (2000) Sample path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times. Appl Math (Warsaw) 27:343–367

    Google Scholar 

  • Venel X (2015) Commutative stochastic games. Math Oper Res 40:403–428

    Article  MathSciNet  MATH  Google Scholar 

  • Vieille N (2002) Stochastic games: recent results. In: Aumann RJ, Hart S (eds) Handbook of Game Theory with Economic Applications, vol 3. North Holland, Amsterdam/London, pp 1833–1850

    Google Scholar 

  • Vigeral G (2013) A zero-sum stochastic game with compact action sets and no asymptotic value. Dyn Games Appl 3:172–186

    Article  MathSciNet  MATH  Google Scholar 

  • Vrieze OJ (1981) Linear programming and undiscounted stochastic games. Oper Res Spektrum 3:29–35

    Article  MATH  Google Scholar 

  • Vrieze OJ (1987) Stochastic games with finite state and action spaces. Mathematisch Centrum Tract, vol 33. Centrum voor Wiskunde en Informatica, Amsterdam

    Google Scholar 

  • Vrieze OJ, Tijs SH, Raghavan TES, Filar JA (1983) A finite algorithm for switching control stochastic games. Oper Res Spektrum 5:15–24

    Article  MATH  Google Scholar 

  • Wessels J (1977) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326–335

    Article  MathSciNet  MATH  Google Scholar 

  • Winston W (1978) A stochastic game model of a weapons development competition. SIAM J Control Optim 16:411–419

    Article  MathSciNet  MATH  Google Scholar 

  • Zachrisson LE (1964) Markov games. In: Dresher M, Shapley LS, Tucker AW (eds) Advances in Game Theory. Princeton University Press, Princeton, pp 211–253

    Google Scholar 

  • Ziliotto B (2016) General limit value in zero-sum stochastic games. Int J Game Theory 45:353–374

    Article  MathSciNet  MATH  Google Scholar 

  • Ziliotto B (2016) Zero-sum repeated games: counterexamples to the existence of the asymptotic value and the conjecture. Ann Probab 44:1107–1133

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We thank Tamer Başar and Georges Zaccour for inviting us to write this chapter and their help. We also thank Eilon Solan, Sylvain Sorin, William Sudderth, and two reviewers for their comments on an earlier version of this survey.

Author information

Authors and Affiliations

  1. Faculty of Pure and Applied Mathematics, Wrocław University of Science and Technology, 50-370, Wybrzeże Wyspiańskiego 27, Wrocław, Poland

    Anna Jaśkiewicz

  2. Faculty of Mathematics, Computer Science and Econometrics, University of Zielona Góra, 65-246, Podgórna 50, Zielona Góra, Poland

    Andrzej S. Nowak

Authors
  1. Anna Jaśkiewicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrzej S. Nowak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Jaśkiewicz .

Editor information

Editors and Affiliations

  1. Coordinated Science Laboratory and Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA

    Tamer Başar

  2. Department of Decision Sciences GERAD, HEC Montréal, Montreal, Québec, Canada

    Georges Zaccour

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Jaśkiewicz, A., Nowak, A.S. (2018). Zero-Sum Stochastic Games. In: Başar, T., Zaccour, G. (eds) Handbook of Dynamic Game Theory. Springer, Cham. https://doi.org/10.1007/978-3-319-44374-4_8

Download citation

Publish with us

Policies and ethics

Search

Navigation