Skip to main content
SpringerLink
Log in

Introduction

  • Chapter
  • First Online:
Nonlinearly Perturbed Semi-Markov Processes

  • 485 Accesses

Abstract

This book is devoted to the study of asymptotic expansions for moment of hitting times, stationary and conditional quasi-stationary distributions, and other functionals, for nonlinearly perturbed semi-Markov processes. The introduction intends to present in informal form the main problems, methods, and algorithms that constitute the content of the book.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abadov, Z.A.: Asymptotical expansions with explicit estimation of constants for exponential moments of sums of random variables defined on a Markov chain and their applications to limit theorems for first hitting times. Candidate of Science Dissertation, Kiev State University (1984)

    Google Scholar 

  • Abbad, M., Filar, J.A.: Algorithms for singularly perturbed Markov control problems: a survey. In: Leondes, C.T. (ed.) Techniques in Discrete-Time Stochastic Control Systems. Control and Dynamic Systems, vol. 73, pp. 257–289. Academic Press, New York (1995)

    Chapter  Google Scholar 

  • Aliev, S.A., Abadov, Z.A.: A new approach for studying asymptotics of exponential moments of sums of random variables connected in Markov chain. I. Trans. Inst. Math. Mech. Nat. Acad, Sci. Azerb. 27(7), 17–32 (2007)

    Google Scholar 

  • Allen, B., Tarnita, C.E.: Measures of success in a class of evolutionary models with fixed population size and structure. J. Math. Biol. 68, 109–143 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Andersson, F., Silvestrov, S.: The mathematics of Internet search engines. Acta Appl. Math. 104, 211–242 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Anisimov, V.V.: Random Processes with Discrete Components, 183 pp. Vysshaya Shkola and Izdatel’stvo Kievskogo Universiteta, Kiev (1988)

    Google Scholar 

  • Anisimov, V.V.: Switching Processes in Queueing Models. Applied Stochastic Methods Series, 345 pp. ISTE, London and Wiley, Hoboken, NJ (2008)

    Google Scholar 

  • Asmussen, S.: Applied Probability and Queues. 2nd edn. Applications of Mathematics, vol. 51. Stochastic Modelling and Applied Probability, xii+438 pp. Springer, New York (2003)

    Google Scholar 

  • Asmussen, S., Albrecher, H.: Ruin Probabilities, 2nd edn. Advanced Series on Statistical Science & Applied Probability, vol. 14, xviii+602 pp. World Scientific, Singapore (2010)

    Google Scholar 

  • Avrachenkov, K.E.: Analytic perturbation theory and its applications. Ph.D. Thesis, University of South Australia (1999)

    Google Scholar 

  • Avrachenkov, K.E.: Singularly perturbed finite Markov chains with general ergodic structure. In: Boel, R., Stremersch, G. (eds.) Discrete Event Systems. Analysis and Control. Kluwer International Series in Engineering and Computer Science, vol. 569, pp. 429–432. Kluwer, Boston (2000)

    Google Scholar 

  • Avrachenkov, K.E., Haviv, M.: Perturbation of null spaces with application to the eigenvalue problem and generalized inverses. Linear Algebra Appl. 369, 1–25 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Avrachenkov, K.E., Haviv, M.: The first Laurent series coefficients for singularly perturbed stochastic matrices. Linear Algebra Appl. 386, 243–259 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Avrachenkov, K.E., Lasserre, J.B.: The fundamental matrix of singularly perturbed Markov chains. Adv. Appl. Probab. 31(3), 679–697 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Avrachenkov, K.E., Filar, J., Haviv, M.: Singular perturbations of Markov chains and decision processes. In: Feinberg, E.A., Shwartz, A. (eds.) Handbook of Markov Decision Processes. Methods and Applications. International Series in Operations Research & Management Science, vol. 40, pp. 113–150. Kluwer, Boston (2002)

    Google Scholar 

  • Avrachenkov, K.E., Filar, J.A., Howlett, P.G.: Analytic Perturbation Theory and Its Applications, xii+372 pp. SIAM, Philadelphia, PA (2013)

    Book  MATH  Google Scholar 

  • Avrachenkov, K., Eshragh, A., Filar, J.A.: On transition matrices of Markov chains corresponding to Hamiltonian cycles. Ann. Oper. Res. 243(1–2), 19–35 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Barbu, V.S., Limnios, N.: Semi-Markov Chains and Hidden Semi-Markov Models Toward Applications. Their Use in Reliability and DNA Analysis. Lecture Notes in Statistics, vol. 191, xiv+224 pp. Springer, New York (2008)

    Google Scholar 

  • Bini, D.A., Latouche, G., Meini, B.: Numerical Methods for Structured Markov Chains. Numerical Mathematics and Scientific Computation. Oxford Science Publications, xii+327 pp. Oxford University Press, New York (2005)

    Google Scholar 

  • Blackwell, D.: Discrete dynamic programming. Ann. Math. Stat. 33, 719–726 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  • Borkar, V.S., Ejov, V., Filar, J.A., Nguyen, G.T.: Hamiltonian Cycle Problem and Markov Chains. International Series in Operations Research & Management Science, vol. 171, xiv+201 pp. Springer, New York (2012)

    Google Scholar 

  • Borovkov, A.A.: Asymptotic Methods in Queuing Theory. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, xi+292 pp. Wiley, Chichester (1984) (English translation of Asymptotic Methods in Queuing Theory. Probability Theory and Mathematical Statistics, 382 pp. Nauka, Moscow (1980))

    Google Scholar 

  • Borovkov, A.A.: Ergodicity and Stability of Stochastic Processes. Wiley Series in Probability and Statistics, vol. 314, xxiv+585 pp. Wiley, Chichester (1998) (Translation from the 1994 Russian original)

    Google Scholar 

  • Cao, W.L., Stewart, W.J.: Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. J. Assoc. Comput. Mach. 32, 702–719 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  • Chatelin, F., Miranker, W.L.: Aggregation/disaggregation for eigenvalue problems. SIAM J. Numer. Anal. 21(3), 567–582 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  • Coderch, M., Willsky, A.S., Sastry, S.S., Castañon, D.A.: Hierarchical aggregation of singularly perturbed finite state Markov processes. Stochastics 8, 259–289 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  • Collet, P., Martínez, S., San Martín, J.: Quasi-Stationary Distributions. Markov Chains, Diffusions and Dynamical Systems. Probability and its Applications, xvi+280 pp. Springer, Heidelberg (2013)

    Google Scholar 

  • Courtois, P.J.: Decomposability: Queueing and Computer System Applications. ACM Monograph Series, xiii+201 pp. Academic Press, New York (1977)

    Google Scholar 

  • Courtois, P.J., Louchard, G.: Approximation of eigen characteristics in nearly-completely decomposable stochastic systems. Stoch. Process. Appl. 4, 283–296 (1976)

    Article  MATH  Google Scholar 

  • Courtois, P.J., Semal, P.: Block decomposition and iteration in stochastic matrices. Philips J. Res. 39(4–5), 178–194 (1984)

    MathSciNet  MATH  Google Scholar 

  • Craven, B.D.: Perturbed Markov processes. Stoch. Models 19(2), 269–285 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Daley, D.J., Gani, J.: Epidemic Modelling: An Introduction. Cambridge Studies in Mathematical Biology, vol. 15, xii+213 pp. Cambridge University Press, Cambridge (1999)

    Google Scholar 

  • Delebecque, F.: A reduction process for perturbed Markov chains. SIAM J. Appl. Math. 43, 325–350 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  • Durrett, R.: Probability Models for DNA Sequence Evolution, xii+431 pp. Springer, New York (2008) (2nd revised edition of Probability Models for DNA Sequence Evolution, viii+240 pp. Springer, New York (2002))

    Book  Google Scholar 

  • Englund, E.: Nonlinearly perturbed renewal equations with applications to a random walk. In: Silvestrov, D., Yadrenko, M., Olenko A., Zinchenko, N. (eds.) Proceedings of the Third International School on Applied Statistics, Financial and Actuarial Mathematics, Feodosiya, 2000. Theory Stoch. Process, 6(22)(3–4), pp. 33–60 (2000)

    Google Scholar 

  • Englund, E.: Nonlinearly perturbed renewal equations with applications. Doctoral Dissertation, Umeå University (2001)

    Google Scholar 

  • Englund, E., Silvestrov, D.S.: Mixed large deviation and ergodic theorems for regenerative processes with discrete time. In: Jagers, P., Kulldorff, G., Portenko, N., Silvestrov, D. (eds.) Proceedings of the Second Scandinavian–Ukrainian Conference in Mathematical Statistics, Vol. I, Umeå, 1997. Theory Stoch. Process. 3(19)(1–2), pp. 164–176 (1997)

    Google Scholar 

  • Engström, C.: PageRank in evolving networks and applications of graphs in natural language processing and biology. Doctoral Dissertation, 217, Mälardalen University, Västerås (2016)

    Google Scholar 

  • Engström, C., Silvestrov, S.: Generalisation of the damping factor in PageRank for weighted networks. In: Silvestrov, D., Martin-Löf, A. (eds.) Modern Problems in Insurance Mathematics, chap. 19 EAA Series, pp. 313–334. Springer, Cham (2014)

    Google Scholar 

  • Engström, C., Silvestrov, S.: PageRank, a look at small changes in a line of nodes and the complete graph. In: Silvestrov, S., Ranc̆ić, M. (eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, chap. 11 Springer Proceedings in Mathematics & Statistics, vol. 179, pp. 223–248. Springer, Cham (2016a)

    Google Scholar 

  • Engström, C., Silvestrov, S.: PageRank, connecting a line of nodes with a complete graph. In: Silvestrov, S., Ranc̆ić, M. (eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, chap. 12 Springer Proceedings in Mathematics & Statistics, vol. 179, pp. 249–274. Springer, Cham (2016b)

    Google Scholar 

  • Feinberg, B.N., Chiu, S.S.: A method to calculate steady-state distributions of large Markov chains by aggregating states. Oper. Res. 35(2), 282–290 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Feller, W.: An Introduction to Probability Theory and Its Applications. Vol. I, xviii+509 pp. Wiley, New York (1968) (3rd edition of An Introduction to Probability Theory and Its Applications. Vol. I, xii+419 pp. Wiley, New York (1950))

    Google Scholar 

  • Gaĭtsgori, V.G., Pervozvanskiy, A.A.: Decomposition and aggregation in problems with a small parameter. Izv. Akad. Nauk SSSR. Tekhn. Kibernet. (1) 33–46 (1983) (English translation in Eng. Cybernetics 2(1), 26–38)

    Google Scholar 

  • Grabski, F.: Semi-Markov Processes: Applications in System Reliability and Maintenance, xiv+255 pp. Elsevier, Amsterdam (2015)

    MATH  Google Scholar 

  • Grassman, W.K., Taksar, M.I., Heyman, D.P.: Regenerative analysis and steady state distributions for Markov chains. Oper. Res. 33, 1107–1116 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  • Gyllenberg, M., Hanski, I.: Single-species metapopulation dynamics: a structured model. Theor. Popul. Biol. 42(1), 35–61 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Gyllenberg, M., Silvestrov, D.S.: Quasi-stationary phenomena in semi-Markov models. In: Janssen, J., Limnios, N. (eds.) Proceedings of the Second International Symposium on Semi-Markov Models: Theory and Applications, Compiègne, 1998, pp. 87–93 (1998)

    Google Scholar 

  • Gyllenberg, M., Silvestrov, D.S.: Quasi-stationary phenomena for semi-Markov processes. In: Janssen, J., Limnios, N. (eds.) Semi-Markov Models and Applications, pp. 33–60. Kluwer, Dordrecht (1999)

    Chapter  Google Scholar 

  • Gyllenberg, M., Silvestrov, D.S.: Nonlinearly perturbed regenerative processes and pseudo-stationary phenomena for stochastic systems. Stoch. Process. Appl. 86, 1–27 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Gyllenberg, M., Silvestrov, D.S.: Quasi-Stationary Phenomena in Nonlinearly Perturbed Stochastic Systems. De Gruyter Expositions in Mathematics, vol. 44, ix+579 pp. Walter de Gruyter, Berlin (2008)

    Google Scholar 

  • Hanen, A.: Théorèmes limites pour une suite de chaînes de Markov. Ann. Inst. H. Poincaré 18, 197–301 (1963)

    MathSciNet  MATH  Google Scholar 

  • Hanski, I.: Metapopulation Ecology. Oxford Series in Ecology and Evolution, 324 pp. Oxford University Press, Oxford (1999)

    Google Scholar 

  • Hanski, I., Gilpin, M.E.: Metapopulation Biology: Ecology, Genetics, and Evolution, 512 pp. Academic Press, London (1997)

    MATH  Google Scholar 

  • Harlamov, B.: Continuous Semi-Markov Processes. Applied Stochastic Methods Series, 375 pp. ISTE, London and Wiley, Hoboken, NJ (2008)

    Google Scholar 

  • Hassin, R., Haviv, M.: Mean passage times and nearly uncoupled Markov chains. SIAM J. Discrete Math. 5, 386–397 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Hassin, R., Haviv, M.: To Queue or not to Queue: Equilibrium Behavior in Queueing Systems. International Series in Operations Research & Management Science, vol. 59, xii+191 pp. Kluwer, Boston (2003)

    Google Scholar 

  • Haviv, M.: An approximation to the stationary distribution of a nearly completely decomposable Markov chain and its error analysis. SIAM J. Algebraic Discrete Methods 7(4), 589–593 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  • Haviv, M.: Aggregation/disaggregation methods for computing the stationary distribution of a Markov chain. SIAM J. Numer. Anal. 24(4), 952–966 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Haviv, M.: An aggregation/disaggregation algorithm for computing the stationary distribution of a large Markov chain. Commun. Stat. Stoch. Models 8(3), 565–575 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Haviv, M.: On censored Markov chains, best augmentations and aggregation/disaggregation procedures. Aggregation and disaggregation in operations research. Comput. Oper. Res. 26(10–11), 1125–1132 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Haviv, M., Ritov, Y.: On series expansions and stochastic matrices. SIAM J. Matrix Anal. Appl. 14(3), 670–676 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  • Haviv, M., Ritov, Y., Rothblum, U.G.: Taylor expansions of eigenvalues of perturbed matrices with applications to spectral radii of nonnegative matrices. Linear Algebra Appl. 168, 159–188 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Hörmander, L.: An Introduction to Complex Analysis in Several Variables, 3rd edn. North-Holland Mathematical Library, vol. 7, xii+254 pp. North-Holland, Amsterdam (1990) (3rd edition of An Introduction to Complex Analysis in Several Variables, x+208 pp. D. Van Nostrand, Princeton (1966))

    Google Scholar 

  • Howard, R.A.: Dynamic Probabilistic Systems, Volume II: Semi-Markov and Decision Processes. Dover Books on Mathematics, 576 pp. Dover, New York (2007) (Reprint of Wiley 1971 edition)

    Google Scholar 

  • Hunter, J.J.: Mixing times with applications to perturbed Markov chains. Linear Algebra Appl. 417(1), 108–123 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Hunter, J.J.: Generalized inverses of Markovian kernels in terms of properties of the Markov chain. Linear Algebra Appl. 447, 38–55 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Janssen, J., Manca, R.: Applied Semi-Markov Processes, xii+309 pp.. Springer, New York (2006)

    MATH  Google Scholar 

  • Janssen, J., Manca, R.: Semi-Markov Risk Models for Finance, Insurance and Reliability, xvii+429 pp. Springer, New York (2007)

    MATH  Google Scholar 

  • Kalashnikov, V.V.: Qualitative Analysis of the Behaviour of Complex Systems by the Method of Test Functions. Series in Theory and Methods of Systems Analysis, 247 pp. Nauka, Moscow (1978)

    Google Scholar 

  • Kalashnikov, V.V.: Mathematical Methods in Queuing Theory. Mathematics and Its Applications, vol. 271, x+377 pp. Kluwer, Dordrecht (1994)

    Google Scholar 

  • Kalashnikov, V.V.: Geometric Sums: Bounds for Rare Events with Applications. Mathematics and Its Applications, vol. 413, ix+265 pp. Kluwer, Dordrecht (1997)

    Google Scholar 

  • Karlin, S., Taylor, H.M.: A First Course in Stochastic Processes, 2nd edn., xvii+557 pp. Academic Press, New York (1975)

    Google Scholar 

  • Kartashov, N.V.: Asymptotic expansions and inequalities in stability theorems for general Markov chains under relatively bounded perturbations. In: Stability Problems for Stochastic Models, Varna, pp. 75–85. VNIISI, Moscow (1985) (English translation in J. Sov. Math. 40(4), 509–518)

    Google Scholar 

  • Kartashov, M.V.: Strong Stable Markov Chains, 138 pp. VSP, Utrecht and TBiMC, Kiev (1996)

    MATH  Google Scholar 

  • Kim, D.S., Smith, R.L.: An exact aggregation/disaggregation algorithm for large scale Markov chains. Nav. Res. Logist. 42(7), 1115–1128 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  • Kokotović, P.V., Phillips, R.G., Javid, S.H.: Singular perturbation modeling of Markov processes. In: Bensoussan, A., Lions, J.L. (eds.) Analysis and Optimization of Systems: Proceedings of the Fourth International Conference on Analysis and Optimization. Lecture Notes in Control and Information Science, vol. 28, pp. 3–15. Springer, Berlin (1980)

    Google Scholar 

  • Korolyuk, V.S.: On asymptotical estimate for time of a semi-Markov process being in the set of states. Ukr. Mat. Zh. 21, 842–845 (1969) (English translation in Ukr. Math. J. 21, 705–710)

    Google Scholar 

  • Korolyuk, V.S.: Stochastic Models of Systems, 208 pp. Naukova Dumka, Kiev (1989)

    Google Scholar 

  • Korolyuk, V.S., Korolyuk, V.V.: Stochastic Models of Systems. Mathematics and Its Applications, vol. 469, xii+185 pp. Kluwer, Dordrecht (1999)

    Google Scholar 

  • Koroliuk, V.S., Limnios, N.: Stochastic Systems in Merging Phase Space, xv+331 pp. World Scientific, Singapore (2005)

    Book  MATH  Google Scholar 

  • Korolyuk, V., Swishchuk, A.: Semi-Markov Random Evolutions. Mathematics and Its Applications, vol. 308, x+310 pp. Kluwer, Dordrecht (1995) (English revised edition of Semi-Markov Random Evolutions, 254 pp. Naukova Dumka, Kiev (1992))

    Google Scholar 

  • Korolyuk, V.S., Turbin, A.F.: On the asymptotic behaviour of the occupation time of a semi-Markov process in a reducible subset of states. Teor. Veroyatn. Mat. Stat. 2, 133–143 (1970) (English translation in Theory Probab. Math. Stat. 2, 133–143)

    Google Scholar 

  • Korolyuk, V.S., Turbin, A.F.: Semi-Markov Processes and its Applications, 184 pp. Naukova Dumka, Kiev (1976)

    MATH  Google Scholar 

  • Korolyuk, V.S., Turbin, A.F.: Mathematical Foundations of the State Lumping of Large Systems, 218 pp. Naukova Dumka, Kiev (1978) (English edition: Mathematical Foundations of the State Lumping of Large Systems. Mathematics and its Applications, vol. 264, x+278 pp. Kluwer, Dordrecht (1993))

    Google Scholar 

  • Korolyuk, V.S., Brodi, S.M., Turbin, A.F.: Semi-Markov processes and their application. Probability Theory. Mathematical Statistics. Theoretical Cybernetics, vol. 11, pp. 47–97. VINTI, Moscow (1974)

    Google Scholar 

  • Korolyuk, V.S., Penev, I.P., Turbin, A.F.: Asymptotic expansion for the distribution of the absorption time of a weakly inhomogeneous Markov chain. In: Korolyuk, V.S. (ed.) Analytic Methods of Investigation in Probability Theory, pp. 97–105. Akad. Nauk Ukr. SSR, Inst. Mat., Kiev (1981)

    Google Scholar 

  • Kovalenko, I.N.: Studies in the Reliability Analysis of Complex Systems, 210 pp. Naukova Dumka, Kiev (1975)

    Google Scholar 

  • Kovalenko, I.N.: Rare events in queuing theory – a survey. Queuing Syst. Theory Appl. 16(1–2), 1–49 (1994)

    MATH  Google Scholar 

  • Kovalenko, I.N., Kuznetsov, N.Yu., Shurenkov, V.M.: Models of Random Processes. A Handbook for Mathematicians and Engineers, 446 pp. CRC, Boca Raton, FL (1996) (A revised edition of the 1983 Russian original)

    Google Scholar 

  • Kovalenko, I.N., Kuznetsov, N.Yu., Pegg, P.A.: Mathematical Theory of Reliability of Time Dependent Systems with Practical Applications. Wiley Series in Probability and Statistics, 316 pp. Wiley, New York (1997)

    Google Scholar 

  • Lande, R., Engen, S., Saether, B-E.: Stochastic Population Dynamics in Ecology and Conservation. Oxford Series and Ecology and Evolution, x+212 pp. Oxford University Press, Oxford (2003)

    Google Scholar 

  • Langville, A.N., Meyer, C.D.: Google’s PageRank and Beyond: The Science of Search Engine Rankings, x+224 pp. Princeton University Press, Princeton, NJ (2006)

    MATH  Google Scholar 

  • Latouche, G., Louchard, G.: Return times in nearly decomposable stochastic processes. J. Appl. Probab. 15, 251–267 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  • Lévy, P.: Processus semi-Markoviens. In: Erven, P., Noordhoff, N.V. (eds.) Proceedings of the International Congress of Mathematicians, Vol. III. Amsterdam, 1954, pp. 416–426. North-Holland, Amsterdam (1954/1956)

    Google Scholar 

  • Limnios, N., Oprişan, G.: Semi-Markov Processes and Reliability. Statistics for Industry and Technology, xii+222 pp. Birkhäuser, Boston (2001)

    Google Scholar 

  • Marek, I., Pultarová, I.: A note on local and global convergence analysis of iterative aggregation-disaggregation methods. Linear Algebra Appl., 413(2–3), 327–341 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Marek, I., Mayer, P., Pultarová, I.: Convergence issues in the theory and practice of iterative aggregation/disaggregation methods. Electron. Trans. Numer. Anal. 35, 185–200 (2009)

    MathSciNet  MATH  Google Scholar 

  • Markushevich, A.I.: Theory of Functions of a Complex Variable. Vol. I, II, III. 2nd edn., xxiii+1138 pp. Chelsea, New York (1977, 1985) (English edition of Theory of Analytic Functions. GITTL, Moscow, 1950)

    Google Scholar 

  • Meshalkin, L.D.: Limit theorems for Markov chains with a finite number of states. Teor. Veroyatn. Primen. 3, 361–385 (1958) (English translation in Theory Probab. Appl. 3, 335–357)

    Google Scholar 

  • Meyer, C.D.: Stochastic complementation, uncoupling Markov chains, and the theory of nearly reducible systems. SIAM Rev. 31(2), 240–272 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability, xxviii+594 pp. Cambridge University Press, Cambridge (2009) (2nd edition of Markov Chains and Stochastic Stability. Communications and Control Engineering Series, xvi+ 548 pp. Springer, London (1993))

    Book  Google Scholar 

  • Miller, B.L., Veinott Jr., A.F.: Discrete dynamic programming with a small interest rate. Ann. Math. Stat. 40, 366–370 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  • Nåsell, I.: Extinction and Quasi-Stationarity in the Stochastic Logistic SIS Model. Lecture Notes in Mathematics, vol. 2022. Mathematical Biosciences Subseries, xii+199 pp. Springer, Heidelberg (2011)

    Google Scholar 

  • Ni, Y.: Perturbed renewal equations with multivariate non-polynomial perturbations. In: Frenkel I., Gertsbakh I., Khvatskin L., Laslo Z., Lisnianski, A. (eds.) Proceedings of the International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management, Beer Sheva, pp. 754–763 (2010a)

    Google Scholar 

  • Ni, Y.: Analytical and numerical studies of perturbed renewal equations with multivariate non-polynomial perturbations. J. Appl. Quant. Methods 5(3), 498–515 (2010b)

    Google Scholar 

  • Ni, Y.: Nonlinearly perturbed renewal equations: asymptotic results and applications. Doctoral Dissertation, 106, Mälardalen University, Västerås (2011)

    Google Scholar 

  • Ni, Y.: Nonlinearly perturbed renewal equations: the non-polynomial case. Teor. Ǐmovirn. Mat. Stat. 84, 111–122 (2012) (Also in Theory Probab. Math. Stat. 84, 117–129)

    Google Scholar 

  • Ni, Y.: Exponential asymptotical expansions for ruin probability in a classical risk process with non-polynomial perturbations. In: Silvestrov, D., Martin-Löf, A. (eds.) Modern Problems in Insurance Mathematics, chap. 6. EAA Series, pp. 67–91. Springer, Cham (2014)

    Google Scholar 

  • Ni, Y., Silvestrov, D., Malyarenko, A.: Exponential asymptotics for nonlinearly perturbed renewal equation with non-polynomial perturbations. J. Numer. Appl. Math. 1(96), 173–197 (2008)

    MATH  Google Scholar 

  • Obzherin, Y.E., Boyko, E.G.: Semi-Markov Models. Control of Restorable Systems with Latent Failures, xiv+199 pp. Elsevier/Academic Press, Amsterdam (2015)

    Google Scholar 

  • Pervozvanskiĭ, A.A., Smirnov, I.N.: An estimate of the steady state of a complex system with slowly varying constraints. Kibernetika (4), 45–51 (1974) (English translation in Cybernetics, 10(4), 603–611)

    Google Scholar 

  • Petersson, M.: Quasi-stationary distributions for perturbed discrete time regenerative processes. Teor. Ǐmovirn. Mat. Stat. 89, 140–155 (2013) (Also in Theor. Probab. Math. Stat. 89, 153–168)

    Google Scholar 

  • Petersson, M.: Asymptotics of ruin probabilities for perturbed discrete time risk processes. In: Silvestrov, D., Martin-Löf, A. (eds.) Modern Problems in Insurance Mathematics, chap. 7, pp. 93–110. EAA Series, Springer, Cham (2014)

    Google Scholar 

  • Petersson, M.: Asymptotic expansions for moment functionals of perturbed discrete time semi-Markov processes. In: Silvestrov, S., Ranc̆ić, M. (eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, chap. 8, pp. 109–130. Springer Proceedings in Mathematics & Statistics, vol. 179. Springer, Cham (2016a)

    Google Scholar 

  • Petersson, M.:. Asymptotics for quasi-stationary distributions of perturbed discrete time semi-Markov processes. In: Silvestrov, S., Ranc̆ić, M. (eds.) Engineering Mathematics II. Algebraic and Analysis Structures for Networks, Data Classification, Optimization and Stochastic Processes, chap. 9 Springer Proceedings in Mathematics & Statistics, vol. 179, pp. 131–150. Springer, Cham (2016b)

    Google Scholar 

  • Petersson, M.: Perturbed discrete time stochastic models. Doctoral Dissertation, Stockholm University (2016c)

    Google Scholar 

  • Phillips, R.G., Kokotović, P.V.: A singular perturbation approach to modeling and control of Markov chains. IEEE Trans. Autom. Control 26, 1087–1094 (1981)

    Article  MATH  Google Scholar 

  • Pinsky, M.A., Karlin, S.: An Introduction to Stochastic Modeling, 4th edn., x+563 pp. Elsevier/Academic Press, Amsterdam (2011)

    Google Scholar 

  • Poliščuk, L.I., Turbin, A.F.: Asymptotic expansions for certain characteristics of semi-Markov processes. Teor. Veroyatn. Mat. Stat. 8, 122–127 (1973) (English translation in Theory Probab. Math. Stat. 8, 121–126)

    Google Scholar 

  • Schweitzer, P.J.: Perturbation theory and finite Markov chains. J. Appl. Probab. 5, 401–413 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  • Schweitzer, P.J.: Dual bounds on the equilibrium distribution of a finite Markov chain. J. Math. Anal. Appl. 126(2), 478–482 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Schweitzer, P.J.: A survey of aggregation-disaggregation in large Markov chains. In: Stewart, W.J. (ed.) Numerical Solution of Markov Chains. Probability: Pure and Applied, vol. 8, pp. 63–88. Dekker, New York (1991)

    Google Scholar 

  • Schweitzer, P.J., Kindle, K.W.: Iterative aggregation for solving undiscounted semi-Markovian reward processes. Commun. Stat. Stoch. Models 2(1), 1–41 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  • Schweitzer, P., Stewart, G.W.: The Laurent expansion of pencils that are singular at the origin. Linear Algebra Appl. 183, 237–254 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  • Schweitzer, P.J., Puterman, M.L., Kindle, K.W.: Iterative aggregation-disaggregation procedures for discounted semi-Markov reward processes. Oper. Res. 33(3), 589–605 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  • Seneta, E.: Finite approximations to infinite non-negative matrices. Proc. Camb. Philol. Soc. 63, 983–992 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  • Seneta, E.: Finite approximations to infinite non-negative matrices. II. Refinements and applications. Proc. Camb. Philol. Soc. 64, 465–470 (1968a)

    Google Scholar 

  • Seneta, E.: The principle of truncations in applied probability. Comment. Math. Univ. Carolinae 9, 237–242 (1968b)

    Google Scholar 

  • Seneta, E.: Nonnegative Matrices. An Introduction to Theory and Applications, x+214 pp. Wiley, New York (1973)

    Google Scholar 

  • Seneta, E.: Iterative aggregation: convergence rate. Econ. Lett. 14(4), 357–361 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  • Seneta, E.: Nonnegative Matrices and Markov Chains. Springer Series in Statistics, xvi+287 pp. Springer, New York (2006) (A revised reprint of 2nd edition of Nonnegative Matrices and Markov Chains. Springer Series in Statistics, xiii+279 pp. Springer, New York (1981))

    Google Scholar 

  • Shurenkov, V.M.: Ergodic Markov Processes. Probability Theory and Mathematical Statistics, 332 pp. Nauka, Moscow (1989)

    Google Scholar 

  • Silvestrov, D.S.: Limit Theorems for Composite Random Functions, 318 pp. Vysshaya Shkola and Izdatel’stvo Kievskogo Universiteta, Kiev (1974)

    Google Scholar 

  • Silvestrov, D.S.: Semi-Markov Processes with a Discrete State Space. Library for the Engineer in Reliability, 272 pp. Sovetskoe Radio, Moscow (1980)

    Google Scholar 

  • Silvestrov, D.S.: Exponential asymptotic for perturbed renewal equations. Teor. Ǐmovirn. Mat. Stat. 52, 143–153 (1995) (English translation in Theory Probab. Math. Stat. 52, 153–162)

    Google Scholar 

  • Silvestrov D.S.: Limit Theorems for Randomly Stopped Stochastic Processes. Probability and Its Applications, xvi+398 pp. Springer, London (2004)

    Google Scholar 

  • Silvestrov, D.S.: Asymptotic expansions for quasi-stationary distributions of nonlinearly perturbed semi-Markov processes. Theory Stoch. Process. 13(1–2), 267–271 (2007)

    MathSciNet  MATH  Google Scholar 

  • Silvestrov D.S.: Nonlinearly perturbed stochastic processes and systems. In: Rykov, V., Balakrishnan, N., Nikulin, M. (eds.) Mathematical and Statistical Models and Methods in Reliability, chap. 2, pp. 19–38. Birkhäuser, Basel (2010)

    Chapter  Google Scholar 

  • Silvestrov, D.S., Abadov, Z.A.: Asymptotic behaviour for exponential moments of sums of random variables defined on exponentially ergodic Markov chains. Dokl. Acad. Nauk Ukr. SSR Ser. A (4), 23–25 (1984)

    MATH  Google Scholar 

  • Silvestrov, D.S., Abadov, Z.A.: Uniform representations of exponential moments of sums of random variables defined on a Markov chain and of distributions of the times of attainment. I. Teor. Veroyatn. Mat. Stat. 45, 108–127 (1991) (English translation in Theory Probab. Math. Stat., 45, 105–120)

    Google Scholar 

  • Silvestrov, D.S., Abadov, Z.A.: Uniform representations of exponential moments of sums of random variables defined on a Markov chain and of distributions of the times of attainment. II. Teor. Veroyatn. Mat. Stat. 48, 175–183 (1993) (English translation in Theory Probab. Math. Stat. 48, 125–130)

    Google Scholar 

  • Silvestrov, D., Manca, R.: Reward algorithms for semi-Markov processes. Method. Comp. Appl. Probab. DOI 10.1007/s11009-017-9559-2, 19 pp. (2017)

    Google Scholar 

  • Silvestrov, D.S., Petersson, M.: Exponential expansions for perturbed discrete time renewal equations. In: Karagrigoriou, A., Lisnianski, A., Kleyner, A., Frenkel, I. (eds.) Applied Reliability Engineering and Risk Analysis. Probabilistic Models and Statistical Inference, chap. 23, pp. 349–362. Wiley, New York (2013)

    Chapter  Google Scholar 

  • Silvestrov, D., Silvestrov, S.: Asymptotic expansions for stationary distributions of perturbed semi-Markov processes. In: Silvestrov, S., Ranc̆ić, M. (eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, chap. 10 Springer Proceedings in Mathematics & Statistics, vol. 179, pp. 151–222. Springer, Cham (2016a)

    Google Scholar 

  • Silvestrov, D., Silvestrov, S.: Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. I, II. Part I: arXiv:1603.04734, 30 pp., Part II: arXiv:1603.04743, 33 pp. (2016b)

    Google Scholar 

  • Silvestrov, D.S., Petersson, M., Hössjer, O.: Nonlinearly perturbed birth-death-type models. Research Report 2016:6. Department of Mathematics, Stockholm University, 63 pp. (2016). arXiv:1604.02295

    Google Scholar 

  • Simon, H.A., Ando, A.: Aggregation of variables in dynamic systems. Econometrica 29, 111–138 (1961)

    Article  MATH  Google Scholar 

  • Smith, W.L.: Regenerative stochastic processes. Proc. R. Soc. Lond. Ser. A 232, 6–31 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  • Stewart, G.W.: Gaussian elimination, perturbation theory, and Markov chains. In: Meyer, C.D., Plemmons, R.J. (eds.) Linear Algebra, Markov Chains, and Queueing Models. IMA Volumes in Mathematics and its Applications, vol. 48, pp. 59–69. Springer, New York (1993a)

    Google Scholar 

  • Stewart, G.W.: On the perturbation of Markov chains with nearly transient states. Numer. Math. 65(1), 135–141 (1993b)

    Google Scholar 

  • Stewart, G.W.: Matrix Algorithms. Vol. I. Basic Decompositions, xx+458 pp. SIAM, Philadelphia, PA (1998)

    Google Scholar 

  • Stewart, G.W.: Matrix Algorithms. Vol. II. Eigensystems, xx+469 pp. SIAM, Philadelphia, PA (2001)

    Google Scholar 

  • Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Computer Science and Scientific Computing, xvi+365 pp. Academic Press, Boston (1990)

    Google Scholar 

  • Stewart, G.W., Zhang, G.: On a direct method for the solution of nearly uncoupled Markov chains. Numer. Math. 59(1), 1–11 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  • Sumita, U., Reiders, M.: A new algorithm for computing the ergodic probability vector for large Markov chains: replacement process approach. Probab. Eng. Inf. Sci. 4, 89–116 (1988)

    Article  Google Scholar 

  • Takács, L.: Some investigations concerning recurrent stochastic processes of a certain type. Magyar. Tud. Akad. Mat. Kutató Int. Kŏzl. 3, 114–128 (1954)

    MathSciNet  Google Scholar 

  • Turbin, A.F.: An application of the theory of perturbations of linear operators to the solution of certain problems that are connected with Markov chains and semi-Markov processes. Teor. Veroyatn. Mat. Stat., 6, 118–128 (1972) (English translation in Theory Probab. Math. Stat. 6, 119–130)

    Google Scholar 

  • Van Doorn, E.A.: Stochastic Monotonicity and Queueing Applications of Birth-Death Processes. Lecture Notes in Statistics, vol. 4, vi+118 pp. Springer, New York (1981)

    Google Scholar 

  • Vantilborgh, H.: Aggregation with an error of O(ε 2). J. Assoc. Comput. Mach. 32(1), 162–190 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  • Veinott Jr., A.F.: Discrete dynamic programming with sensitive discount optimality criteria. Ann. Math. Stat. 40, 1635–1660 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  • Volker, N.: Semi-Markov Processes. Scientific Paperbacks, Mathematics/Physics Series, vol. 260, 162 pp. Akademie-Verlag, Berlin (1980)

    Google Scholar 

  • Whitt, W.: Stochastic-Process Limits. An Introduction to Stochastic-Process Limits and Their Application to Queues. Springer Series in Operations Research, xxii+601 pp. Springer, New York (2002)

    Google Scholar 

  • Yin, G., Zhang, Q.: Discrete-time singularly perturbed Markov chains. In: Yao, D.D., Zhang, H., Zhou, X.Y. (eds.) Stochastic Modeling and Optimization, pp. 1–42. Springer, New York (2003)

    Google Scholar 

  • Yin, G.G., Zhang, Q.: Discrete-Time Markov Chains. Two-Time-Scale Methods and Applications. Stochastic Modelling and Applied Probability, xix+348 pp. Springer, New York (2005)

    Google Scholar 

  • Yin, G.G., Zhang, Q.: Continuous-Time Markov Chains and Applications. A Two-Time-Scale Approach. Stochastic Modelling and Applied Probability, vol. 37, xxii+427 pp. Springer, New York (2013) (2nd revised edition of Continuous-Time Markov Chains and Applications. A Singular Perturbation Approach. Applications of Mathematics, vol. 37, xvi+349 pp. Springer, New York (1998))

    Google Scholar 

  • Yin, G., Zhang, Q., Yang, H., Yin, K.: Discrete-time dynamic systems arising from singularly perturbed Markov chains. In: Proceedings of the Third World Congress of Nonlinear Analysts, Part 7, Catania, 2000. Nonlinear Analysis, vol. 47(7), pp. 4763–4774 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Stockholm University, Stockholm, Sweden

    Dmitrii Silvestrov

  2. Division of Applied Mathematics, School of Education, Culture and Communication (UKK), Mälardalen University, Västerås, Sweden

    Sergei Silvestrov

Authors
  1. Dmitrii Silvestrov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergei Silvestrov
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 The Author(s)

About this chapter

Cite this chapter

Silvestrov, D., Silvestrov, S. (2017). Introduction. In: Nonlinearly Perturbed Semi-Markov Processes. SpringerBriefs in Probability and Mathematical Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-60988-1_1

Download citation

Publish with us

Policies and ethics

Search

Navigation