Abstract
Finite state continuous time Markov processes with weak interactions are modeled as singularly perturbed systems. Aggregate states are obtained using a grouping algorithm. Two-time scale expansions simplify cost equations and lead to decentralized optimization algorithms.
This work was supported in part by the Joint Services Electronics Program under Contract N00014-79-C-0424, in part by the National Science Foundation under Grant ECS-79-19396, and in part by the U. S. Department of Energy under Contract DE-AC01-79-ET29243.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
F. Delebecque and J. P. Quadrat, "Contribution of Stochastic Control Singular Perturbation Averaging Team Theories to an Example of Large-Scale Systems: Management of Hydropower Production," IEEE Trans. on Automatic Control, Vol. AC-23, No. 2, April 1978, pp. 209–222.
F. Delebecque and J. P. Quadrat, "Optimal Control of Markov Chains Admitting Strong and Weak Interactions," to appear in Automatica.
P. J. Courtois, Decomposability: Queueing and Computer System Applications, Academic Press, New York, 1977.
C. H. Sauer and K. M. Chandy, "Approximate Solution of Queueing Models," Computer, April 1980, pp. 25–32.
W. J. Stewart, "A Comparison of Numerical Techniques in Markov Modeling," Comm. ACM, Vol. 21, No. 2, February 1978, pp. 144–151.
A. A. Pervozvanskii and I. N. Smirnov, "Stationary State Evaluation for a Complex System with Slowly Varying Couplings," translation from Kybernetika, No. 4, July, August 1974, pp. 45–51.
V. G. Gaitsgori and A. A. Pervozvanskii, "Aggregation of States in a Markov Chain with Weak Interactions," translation from Kybernetika, No. 3, May–June 1975, pp. 91–98.
A. A. Pervozvanskii and V. G. Gaitsgori, "Suboptimization, Decomposition and Aggregation," Proceedings of the Seventh IFAC World Congress, Helsinki, Finland, 1978.
A. A. Pervozvanskii and V. G. Gaitsgori, Decomposition, Aggregation and Approximate Optimization (in Russian), Nauka, Moscos, 1979.
F. Delebecque and J. P. Quadrat, "The Optimal Cost Expansion of Finite Controls Finite States, Markov Chains with Weak and Strong Interactions," to appear in Mathematics of Operations Research.
B. Avramovic, P. V. Kokotovic, J. R. Winkelman, and J. H. Chow, "Area Decomposition for Electromechanical Models of Power Systems," Automatica, November 1980.
P. V. Kokotovic, R. E. O'Malley, Jr., and P. Sannuti, "Singular Perturbations and Order Reduction in Control Theory—An Overview," Automatica, Vol. 12, 1976, pp. 123–132.
V. F. Butuzov, A. B. Vasil'eva, and M. V. Fedoryvk, "Asymptotic Methods in the Theory of Ordinary Differential Equations," Progress in Mathematics, R. V. Gamkrelidze, Ed., Planum Press, New York, pp. 1–82.
K.-K. D. Young, P. V. Kokotovic, and V. I. Utkin, "A Singular Perturbation Analysis of High Gain Feedback Systems," IEEE Trans. on Automatic Control, Vol. AC-22, 1977, pp. 931–938.
P. V. Kokotovic, J. J. Allemong, J. R. Winkelman, and J. H. Chow, "Singular Perturbations and Iterative Separation of Time Scales," Automatica, Vol. 16, No. 1, January 1980, pp. 23–34.
R. G. Phillips, "Reduced Order Modeling and Control of Two-Time-Scale Discrete Systems," Int. J. of Control, Vol. 31, No. 4, 1980, pp. 765–780.
S. H. Javid, "Multi-Time Methods in Order Reduction and Control of Discrete Systems," Proceedings of the Thirteenth Asilomar Conference on Circuits, Systems, and Computers, Pacific Grove, Calif., November 5–7, 1979.
R. A. Howard, Dynamic Programming and Markov Processes, Wiley, New York, 1960.
R. G. Phillips, "Decomposition of Tiome-Scales in Linear Systems Using Dominant Eigenspace Power Iterations and Matched Asymptotic Expansions," Report DC-31, Coordinated Science Lab., Univ. of Illinois, Urbana, October 1979.
H. Kushner, Introduction to Stochastic Control, Holt, Rinehart, and Winston, 1971.
D. P. Bertsekas, Dynamic Programming and Stochastic Control, Academic Press, New York, Vol. 125, 1977.
D. J. White, "Dynamic Programming, Markov Chains, and the Method of Successive Approximations," J. Math. Analysis and Applications, Vol. 6, 1963, pp. 373–376.
P. Varaiya, "Optimal and Suboptimal Stationary Controls for Markov Chains," IEEE Trans. on Automatic Control, Vol. AC-23, 1979, pp. 388–394.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1980 Springer-Veralg
About this paper
Cite this paper
Kokotovic, P.V., Phillips, R.G., Javid, S.H. (1980). Singular perturbation modeling of Markov processes. In: Bensoussan, A., Lions, J.L. (eds) Analysis and Optimization of Systems. Lecture Notes in Control and Information Sciences, vol 28. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0004030
Download citation
DOI: https://doi.org/10.1007/BFb0004030
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-10472-8
Online ISBN: 978-3-540-38489-2
eBook Packages: Springer Book Archive