Abstract
This paper deals with the infinite horizon linear quadratic (LQ) differential games for discrete-time stochastic systems with both state and control dependent noise. The Popov-Belevitch-Hautus (PBH) criteria for exact observability and exact detectability of discrete-time stochastic systems are presented. By means of them, we give the optimal strategies (Nash equilibrium strategies) and the optimal cost values for infinite horizon stochastic differential games. It indicates that the infinite horizon LQ stochastic differential games are associated with four coupled matrix-valued equations. Furthermore, an iterative algorithm is proposed to solve the four coupled equations. Finally, an example is given to demonstrate our results.
Similar content being viewed by others
References
R. Issacs. Differential Games. New York: Wiley, 1965.
E. Dockner, S. Jorgensen, N. Van Long. Differential Games in Economics and Management Science. London: Cambridge University Press, 2000.
R. Cellini, L. Lambertini. A differential game approach to investment in product differentiation. Journal of Economic Dynamics and Control, 2002, 27(1): 51–62.
J. Engwerda. LQ Dynamic Optimization and Differential Games. Chichester: John Wiley and Sons, 2005.
Y. C. Ho, A. E. Bryson, S. Baron. Differential games and optimal pursuit-evasion strategies. IEEE Transactions on Automatic Control, 1965, 10(4): 385–389.
A. W. Starr, Y. C. Ho. Nonzero-sum differential games. Journal of Optimization Theory and Applications, 1969, 3(3): 184–206.
N. J. Krikelis, Z. V. Rekasius. On the solution of the optimal linear control problems under conflict of interest. IEEE Transactions on Automatic Control, 1971, 16(2): 140–147.
W. A. Van Den Broek, J. Engwerda, J. M. Schumacher. Robust equilibria in indefinite linear-quadratic differential games. Journal of Optimization Theory and Applications, 2003, 119(3): 565–595.
A. EI Bouhtouri, D. Hinrichsen, A. J. Pritchard. H∞-type control for discrete-time stochastic systems. International Journal of Robust and Nonlinear Control, 1999, 9(13): 923–948.
Y. Huang, W. Zhang, H. Zhang. Infinite horizon linear quadratic optimal control for discrete-time stochastic systems. Asian Journal of Control, 2008, 10(5): 608–615.
L. Qian, Z. Gajic. Variance minimization stochastic power control in CDMA systems. IEEE Transactions on Wireless Communications, 2006, 5(1): 193–202.
Z. Lin, Y. Lin, W. Zhang. A unified design for state and output feedback H ∞ control of nonlinear stochastic Markovian jump systems with state and disturbance-dependent noise. Automatica, 2009, 45(12): 2955–2962.
J. L. Willems, J. C. Willems. Feedback stabilizability for stochastic systems with state and control dependent noise. Automatica, 1976, 12(3): 277–283.
A. E. Bashirov, K. R. Kerimov. On controllability conception for stochastic systems. SIAM Journal on Control and Optimization, 1997, 35(2): 384–398.
W. Zhang, B. Chen. On stabilizability and exact observability of stochastic systems with their applications. Automatica, 2004, 40(1): 87–94.
B. Chen, W. Zhang. Stochastic H2/H∞ control with state-dependent noise. IEEE Transactions on Automatic Control, 2004, 49(1): 45–57.
Z. Li, Y. Wang, B. Zhou. Detectability and observability of discretetime stochastic systems and their applications. Automatica, 2009, 45(5): 1340–1346.
H. Sun, M. Li, W. Zhang. Stability and stabilization of stochastic systems with multiplicative noise. International Journal of Control, Automation and Systems, 2011, 9(2): 211–217.
H. Sun, M. Li, W. Zhang. Linear-quadratic stochastic differential game: infinite-time case. ICIC Express Letters, 2011, 5(4B): 1449–1454.
H. Sun, L. Jiang. Linear infinite horizon quadratic differential games for stochastic systems: discrete-time case. Proceedings of the 23rd Chinese Control and Decision Conference, Mianyang: IEEE, 2011: 1733–1737.
M. A. Rami, X. Chen, J. B. Moore, et al. Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls. IEEE Transactions on Automatic Control, 2001, 46(3): 428–440.
W. Zhang, Y. Huang, H. Zhang. Stochastic H2/H∞ control for discrete-time systems with state and disturbance dependent noise. Automatica, 2007, 43(3): 513–521.
G. Tang, H. Sun, H. Pang. Approximately optimal tracking control for discrete time-delay systems with disturbances. Progress in Natural Science, 2008, 18(2): 225–231.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Natural Science Foundation of China (No. 61174078), the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20103718110006), and A Project of Shandong Province Higher Educational Science and Technology Program (No. J12LN14).
Huiying SUN received her M.S. degree from Qingdao University of Science and Technology and Ph.D. degree from the Ocean University of China, in 2004 and 2007, respectively. She is currently an associate professor of Shandong University of Science and Technology. Her main research interests include linear optimal control and stochastic systems.
Liuyang JIANG received her B.S. degree in Electrical Engineering from Qingdao Technological University in 2004. Since September 2009, she has been pursuing her M.S. degree in the Shandong University of Science and Technology. Her main research interests include linear optimal control and stochastic systems.
Weihai ZHANG received his M.S. degree from Hangzhou University, and Ph.D. degree from Zhejiang University, Hangzhou, China, in 1994 and 1998, respectively. From August 1998 to May 2001, he worked at Shandong Institute of Light Industry as an associate professor. He was a postdoctoral researcher from May 2001 to July 2003 at National Tsinghua University, Hsinchu, Taiwan. He is currently a professor of Shandong University of Science and Technology. His research interests include linear and nonlinear stochastic optimal control, robust H-infinity control and stochastic stability.
Rights and permissions
About this article
Cite this article
Sun, H., Jiang, L. & Zhang, W. Infinite horizon linear quadratic differential games for discrete-time stochastic systems. J. Control Theory Appl. 10, 391–396 (2012). https://doi.org/10.1007/s11768-012-1004-z
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11768-012-1004-z