Abstract
We present a simulation-based algorithm to compute the average reward of singulary perturbed Markov Reward Processes (SPMRPs) with large scale state spaces, which depend on some sets of parameters. Compared with the original algorithm applied on these problems of general Markov Reward Processes (MRPs), our algorithm aims to obtain a faster pace in singularly perturbed cases. This algorithm relies on the special structure of singularly perturbed Markov processes, evolves along a single sample path, and hence can be applied on-line.
This work was supported by National Natural Science Foundation of China under Grant(60274012) and Natural Science Foundation of Anhui Province under Grant(01042308).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Marbach, P., Tsitsiklis, J.N.: Simulation-Based Optimization of Markov Reward Processes. IEEE Transactions on Automatic Control 46(2), 191–209 (2001)
Yin, G., Zhang, Q.: Singularly Peturbed Discrete-Time Markov Chains. SIAM Journal appl. math. 61(3), 834–854 (2000)
Abbad, M., Filar, J.A.: Perturbation and Stability Theory for Markov Control Problems. IEEE Transactions on Automatic Control 37(9), 1415–1420 (1992)
Abbad, M., Filar, J.A.: Algorithms for Singularly Perturbed Limiting Average Markov Control Problem. IEEE Transactions on Automatic Control 37(9), 1421–1425 (1992)
Yin, G., Zhang, Q., Badowski, G.: Discrete-Time Singularly Perturbed Markov Chain: Aggregation, Occupation Measures, and Switching Diffusion Limit. Adv. Appl. Prob. 35, 449–476 (2003)
Delebecque, F.: A Reduction Process for Perturbed Markov Chains. SIAM Journal Appl. Math. 43(2), 325–350 (1983)
Liu, R.H., Zhang, Q., Yin, G.: Singularly Perturbed Markov Decision Processes in Discrete Time. In: Proceedings of the 40th IEEE Conference on Decision and Control, December 2001, vol. 3, pp. 2119–2124 (2001)
Hao, T., Xi, H.S., Yin, B.Q.: Performance Optimization of Continuous-Time Markov Control Processes Based on Performance Potentials. International Journal of Systems Science 34(1), 63–71 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, D., Xi, H., Yin, B. (2005). Simulation-Based Optimization of Singularly Perturbed Markov Reward Processes with States Aggregation. In: Huang, DS., Zhang, XP., Huang, GB. (eds) Advances in Intelligent Computing. ICIC 2005. Lecture Notes in Computer Science, vol 3645. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11538356_14
Download citation
DOI: https://doi.org/10.1007/11538356_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28227-3
Online ISBN: 978-3-540-31907-8
eBook Packages: Computer ScienceComputer Science (R0)