Abstract
This paper studies some aspects of perturbation theory applied to Markov chains. In the first part we introduce the notion of agregated chain and show how these chains arise in the context of perturbation and time scales. In the second part, we study some applications of perturbed Markov chains to the Reliability of large scale repairable systems. In the third part we give some applications to optimal control.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
P. BERNHARD. On singular implicit linear dynamical systems, Siam J of Control and Opt. vol 20 no 5 sept 82.
M. CODERCH, A.S. WILLSKI, S.S. SASTRY, D.A. CASTANON. Hierarchical aggregation of linear systems with multiple time scales, MIT Report LIDS-P-1187, mars 1982.
M. CODERCH, A.S. WILLSKY, S.S. SASTRY. Hierarchical aggregation of singulary perturbed finite state Markov chains submitted to stochastics.
P.J. COURTOIS. Decomposability, ACM Monograph Series, Academic Press, 1977.
F. DELEBECQUE. A reduction process for pertubed Markov chains, a paraître SIAM J. of applied math. to appear.
F. DELEBECQUE, J.P. QUADRAT. Optimal control of Markov chains admitting strong and weak interactions, Automatica, Vol. 17, no 2, pp. 281–296, 1981.
F. DELEBECQUE, J.P. QUADRAT. The optimal cost expansion of finite controls finite states Markov chains with weak and strong interactions. Analysis and optimization of systems, Lecture Notes an control and Inf. Science 28 Springer Verlag, 1980.
A.A. PERVOZVANSKII, A.V. GAITSGORI. Decomposition aggregation and approximate optimization en Russe, Nauka, Moscou, 1979.
T. KATO. Perturbation theory for linear operator, Springer Verlag, 1976.
B.L. MILLER, A.F. VEINOTT. Discrete dynamic programming with small interest rate. An. math. stat. 40, 1969, pp. 366–370.
R. PHILIPS, P. KOKOTOVIC. A singular perturbation approach to modelling and control of Markov, chains IEEE A.C. Bellman issue, 1981.
H. SIMON, A. ANDO. Aggregation of variables in dynamic systems, Econometrica, 29, 111–139, 1961.
J. KEMENY, L. SNELL. Finite Markov chains, Van Nostrand, 1960.
O. MURON. Evaluation de politiques de maintenance pour un système complexe, RIRO, vol. 14, no 3, pp. 265–282, 1980.
S.L. CAMBELL, C.D. MEYER jr. Generalized inverses of linear transformations. Pitman, London, 1979.
TKIOUAT. Thèse Rabat à paraitre.
J.P. QUADRAT. Commande optimale de chaines de Markov perturbées Outils et Modèles Math. pour l'automatique... t3 edition CNRS 1983.
J.P. QUADRAT Optimal control of perturbed, Markov chain the multitime scale case. Singular pertubation in systems and control. CISM courses and lectures no 280, Springer Verlag 82.
F. DELEBECQUE, J.P. QUADRAT. Contribution of stochastic control, team theory and singular perturbation to an example of large scale systems: Management of hydropower production. IEEE AC avril 1978.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1987 Springer-Verlag
About this chapter
Cite this chapter
Delebecque, F., Muron, O., Quadrat, J.P. (1987). Singular perturbation of Markov chains. In: Kokotovic, P.V., Bensoussan, A., Blankenship, G.L. (eds) Singular Perturbations and Asymptotic Analysis in Control Systems. Lecture Notes in Control and Information Sciences, vol 90. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0007179
Download citation
DOI: https://doi.org/10.1007/BFb0007179
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-17362-5
Online ISBN: 978-3-540-47440-1
eBook Packages: Springer Book Archive