Skip to main content
SpringerLink
Log in

Markovian Models for Performance and Dependability Evaluation

  • Chapter
  • First Online:
Lectures on Formal Methods and PerformanceAnalysis (EEF School 2000)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2090))

Included in the following conference series:

Abstract

Markovian models have been used for about a century now for the evaluation of the performance and dependability of computer and communication systems. In this paper, we give a concise overview of the most widely used classes of Markovian models, their solution and application.

After a brief introduction to performance and dependability evaluation in general, we introduce discrete-time Markov chains, continuous-time Markov chains and semi-Markov chains. Stepwisely, we develop the main equations that govern the steady-state and the transient behaviour of such Markov chains. We thereby emphasise on intuitively appealing explanations rather than on mathematical rigor. The relation between the various Markov chain types is explained in detail. Then, we discuss means to numerically solve the systems of linear equations (both direct and iterative ones) and the systems of differential equations that arise when solving for the steady-state and transient behaviour of Markovian models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Aziz, K. Sanwal, V. Singhal, and R. Brayton. Verifying continuous time Markov chains. In R. Alur and T. Henzinger, editors, Lecture Notes in Computer Science 1102, pages 269–276, 1996.

    Google Scholar 

  2. A. Aziz, V. Singhal, F. Balarin, R. Brayton, and A. Sangiovanni-Vincentelli. It usually works: the temporal logic of stochastic systems. In P. Wolper, editor, Lecture Notes in Computer Science 939, pages 155–165, 1995.

    Google Scholar 

  3. C. Baier, B.R. Haverkort, J.-P. Katoen, and H. Hermanns. On the logical characterisation of performability properties. In U. Montanari, J.D.P. Rolin, and E. Welzl, editors, Lecture Notes in Computer Science 1853, pages 780–792, 2000.

    Google Scholar 

  4. C. Baier, B.R. Haverkort, J.-P. Katoen, and H. Hermanns. Model checking continuous-time Markov chains by transient analysis. In E.A. Emerson and A.P. Sistla, editors, Lecture Notes in Computer Science 1855, pages 358–372, 2000.

    Google Scholar 

  5. C. Baier, J.-P. Katoen, and H. Hermanns. Approximate symbolic model checking of continuous-time Markov chains. In J.C.M. Baeten and S. Mauw, editors, Lecture Notes in Computer Science 1664, pages 146–161, 1999.

    Google Scholar 

  6. G. Balbo. Introduction to stochastic Petri nets. This volume.

    Google Scholar 

  7. A. Bell and B.R. Haverkort. Serial and parallel out-of-core solution of linear systems arising from generalised stochastic Petri net models. In Proceedings High Performance Computing 2001. Society for Computer Simulation, 2001.

    Google Scholar 

  8. M. Bernardo, R. Cleaveland, S. Sims, and W. Stewart. TwoTowers: a tool integrating functional and performance analysis of concurrent systems. In Proceedings FORTE/PSTV 1998, pages 457–467, 1998.

    Google Scholar 

  9. G. Bolch, S. Greiner, H. de Meer, and K.S. Trivedi. Queueing Networks and Markov Chains. John Wiley & Sons, 1998.

    Google Scholar 

  10. E. Brinksma and H. Hermanns. Process algebra and Markov chains. This volume.

    Google Scholar 

  11. S. Caselli, G. Conte, and P. Marenzoni. Parallel state space exploration for GSPN models. In G. De Michelis and M. Diaz, editors, Applications and Theory of Petri Nets 1995, volume 935 of Lecture Notes in Computer Science, pages 181–200. Springer-Verlag, 1995.

    Google Scholar 

  12. G. Chiola. A software package of the analysis of generalized stochastic Petri nets. In Proceedings of the 1st International Workshop on Timed Petri Nets, pages 136–143, Torino, Italy, July 1985. IEEE Computer Society Press.

    Google Scholar 

  13. G. Ciardo. Distributed and structured analysis approaches to study large and complex systems. This volume.

    Google Scholar 

  14. G. Ciardo, J. Gluckman, and D. Nicol. Distributed state space generation of discrete-state stochastic models. INFORMS Journal of Computing, 10(1):82–93, 1998.

    Article  Google Scholar 

  15. G. Ciardo and A.S. Miner. Storage alternatives for large structured state spaces. In R. Marie, B. Plateau, M. Calzarossa, and G. Rubino, editors, Computer Performance Evaluation, Lecture Notes in Computer Science 1245, pages 44–57. Springer Verlag, 1997.

    Chapter  Google Scholar 

  16. G. Ciardo, J. Muppala, and K.S. Trivedi. SPNP: Stochastic Petri net package. In Proceedings of the 3rd International Workshop on Petri Nets and Performance Models, pages 142–151. IEEE Computer Society Press, 1989.

    Google Scholar 

  17. E.M. Clarke, O. Grumberg, and D. Peled. Model checking. MIT Press, 1999.

    Google Scholar 

  18. N.M. van Dijk. Queueing Networks and Product Form: A Systems Approach. John Wiley & Sons, 1993.

    Google Scholar 

  19. E.M. Clarke, E.A. Emerson, and A.P. Sistla. Automatic verification of finite-state concurrent systems using temporal logic specifications. ACM Transactions on Programming Languages and Systems, 8(2):244–263, 1986.

    Article  MATH  Google Scholar 

  20. A.K. Erlang. Solution of some problems in the theory of probabilities of significance in automatic telephone exchanges. The Post Office Electrical Engineer’s Journal, 10:189–197, 1917.

    Google Scholar 

  21. B.L. Fox and P.W. Glynn. Computing Poisson probabilities. Communications of the ACM, 31(4):440–445, 1988.

    Article  MathSciNet  Google Scholar 

  22. K.A. Frenkel. Allan L. Scherr-Big Blue’s time-sharing pioneer. Communications of the ACM, 30(10):824–828, 1987.

    Article  Google Scholar 

  23. A. Goyal, S.S. Lavenberg, and K.S. Trivedi. The system availability estimator. Annals of Operations Research, 8:285–306, 1987.

    Article  Google Scholar 

  24. W.K. Grassmann. Finding transient solutions in Markovian event systems through randomization. In W.J. Stewart, editor, Numerical Solution of Markov Chains, pages 357–371. Marcel Dekker, 1991.

    Google Scholar 

  25. D. Gross and D.R. Miller. The randomization technique as a modeling tool and solution procedure for transient Markov processes. Operations Research, 32(2):343–361, 1984.

    MATH  MathSciNet  Google Scholar 

  26. A.L. Hageman and D.M. Young. Applied Iterative Methods. Academic Press, 1981.

    Google Scholar 

  27. B.R. Haverkort. Performance of Computer Communication Systems: A Model-Based Approach. John Wiley & Sons, 1998.

    Google Scholar 

  28. B.R. Haverkort, A. Bell, and H. Bohnenkamp. On the efficient sequential and distributed generation of very large Markov chains from stochastic Petri nets. In Proceedings of the 8th International Workshop on Petri Nets and Performance Models, pages 12–21. IEEE Computer Society Press, 1999.

    Google Scholar 

  29. B.R. Haverkort, R. Marie, G. Rubino, and K.S. Trivedi (editors). Performability Modelling: Techniques and Tools. John Wiley & Sons, 2001.

    Google Scholar 

  30. B.R. Haverkort and I.G. Niemegeers. Performability modelling tools and techniques. Performance Evaluation, 25:17–40, 1996.

    Article  MATH  Google Scholar 

  31. B.R. Haverkort and K.S. Trivedi. Specification and generation of Markov reward models. Discrete-Event Dynamic Systems: Theory and Applications, 3:219–247, 1993.

    Article  MATH  Google Scholar 

  32. H. Hermanns, U. Herzog, U. Klehmet, V. Mertsiotakis, and M. Siegle. Compositional performance modelling with the tipptool. Performance Evaluation, 39:5–35, 2000.

    Google Scholar 

  33. H. Hermanns, J.-P. Katoen, J. Meyer-Kayser, and M. Siegle. A Markov chain model checker. In S. Graf and M. Schwartzbach, editors, Lecture Notes in Computer Science 1785, pages 347–362. Springer-Verlag, 2000.

    Google Scholar 

  34. J. Hillston. A Compositional Approach to Performance Modelling. PhD thesis, University of Edinburgh, 1994.

    Google Scholar 

  35. J. Hillston. Exploiting structure in solution: decomposing compositional models. This volume.

    Google Scholar 

  36. A.S. Hornby. Oxford Advanced Learner’s Dictionary of Current English. Oxford University Press, 1974.

    Google Scholar 

  37. R.A. Howard. Dynamic Probabilistic Systems; Volume I: Markov models. John Wiley & Sons, 1971.

    Google Scholar 

  38. R.A. Howard. Dynamic Probabilistic Systems; Volume II: Semi-Markov and decision processes. John Wiley & Sons, 1971.

    Google Scholar 

  39. A. Jensen. Markov chains as an aid in the study of Markov processes. Skand. Aktuarietidskrift, 3:87–91, 1953.

    Google Scholar 

  40. K. Kant. Introduction to Computer System Performance Evaluation. McGraw-Hill, 1992.

    Google Scholar 

  41. J.-P. Katoen. Concepts, Algorithms and Tools for Model Checking. Universität Erlangen-Nürnberg, 1999.

    Google Scholar 

  42. J.G. Kemeny and J.L. Snell. Finite Markov chains. Van Nostrand, Princeton, 1960.

    MATH  Google Scholar 

  43. D.G. Kendall. Some problems in the theory of queues. Journal of the Royal Statistical Society, Ser. B, 13:151–185, 1951.

    MATH  MathSciNet  Google Scholar 

  44. L. Kleinrock. Queueing Systems; Volume 1: Theory. John Wiley & Sons, 1975.

    Google Scholar 

  45. L. Kleinrock. Queueing Systems; Volume 2: Computer Applications. John Wiley & Sons, 1976.

    Google Scholar 

  46. A.N. Kolmogorov. Anfangsgrü;nde der Theorie der Markoffschen Ketten mit unendlichen vielen möglichen Zuständen. Mat. Sbornik N.S., pages 607–610, 1936.

    Google Scholar 

  47. U. Krieger, B. Müller-Clostermann, and M. Sczittnick. Modelling and analysis of communication systems based on computational methods for Markov chains. IEEE Journal on Selected Areas in Communications, 8(9):1630–1648, 1990.

    Article  Google Scholar 

  48. V.G. Kulkarni. Modeling and Analysis of Stochastic Systems. Chapman & Hall, London, Glasgow, Weinheim, 1995.

    MATH  Google Scholar 

  49. A.A. Markov. Investigations of an important case of dependent trails. Izvestia Acad. Nauk VI, Series I (in Russian), 61, 1907.

    Google Scholar 

  50. J.F. Meyer. On evaluating the performability of degradable computing systems. IEEE Transactions on Computers, 29(8):720–731, 1980.

    Article  MATH  Google Scholar 

  51. J.F. Meyer. Closed-form solutions of performability. IEEE Transactions on Computers, 31(7):648–657, 1982.

    Article  Google Scholar 

  52. C. Moler and C.F. van Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Review, 20(4):801–835, 1978.

    Article  MATH  MathSciNet  Google Scholar 

  53. A.P.A. van Moorsel. Performability Evaluation Concepts and Techniques. PhD thesis, University of Twente, 1993.

    Google Scholar 

  54. A.P.A. van Moorsel and B.R. Haverkort. Probabilistic evaluation for the analytical solution of large Markov models: Algorithms and tool support. Microelectronics and Reliability, 36(6):733–755, 1996.

    Article  Google Scholar 

  55. A.P.A. van Moorsel and W.H. Sanders. Adaptive uniformization. Stochastic Models, 10(3):619–648, 1994.

    Article  MATH  MathSciNet  Google Scholar 

  56. B. Müller-Clostermann. NUMAS-a tool for the numerical analysis of computer systems. In D. Potier, editor, Proceedings of the International Conference on Modelling Techniques and Tools for Performance Analysis, pages 141–154. North-Holland, 1985.

    Google Scholar 

  57. J.K. Muppala and K.S. Trivedi. Numerical transient solution of finite Markovianqueueing systems. In U. Bhat, editor, Queueing and Related Models. Oxford UniversityPress, 1992

    Google Scholar 

  58. M.F. Neuts. Matrix Geometric Solutions in Stochastic Models: An Algorithmic Approach. Johns Hopkins University Press, 1981.

    Google Scholar 

  59. M.A. Qureshi and W.H. Sanders. A new methodology for calculating distributions of reward accumulated during a finite interval. In Proceedings of the 26th Symposium on Fault-Tolerant Computer Systems (Sendai, Japan, June 1996), pages 116–125. IEEE Computer Society Press, 1996.

    Google Scholar 

  60. W.H. Sanders and J.-F Meyer. Stochastic activity networks: Formal definitions and concepts. This volume.

    Google Scholar 

  61. W.H. Sanders and J.F. Meyer. Reduced-base model construction for stochastic activity networks. IEEE Journal on Selected Areas in Communications, 9(1):25–36, 1991.

    Article  Google Scholar 

  62. W.H. Sanders, W.D. Obal, M.A. Qureshi, and F.K. Widnajarko. The UltraSAN modeling environment. Performance Evaluation, 24:89–115, 1995.

    Article  MATH  Google Scholar 

  63. E. de Souza e Silva and H.R. Gail. Calculating availability and performability measures of repairable computer systems using randomization. Journal of the ACM, 36(1):171–193, 1989.

    Article  MATH  Google Scholar 

  64. E. de Souza e Silva and H.R. Gail. Performability analysis of computer systems: from model specification to solution. Performance Evaluation, 1:157–196, 1992.

    Article  Google Scholar 

  65. W.J. Stewart. MARCA: Markov chain analyzer. a software package for Markov modeling. In W.J. Stewart, editor, Numerical Solution of Markov Chains, pages 37–62. Marcel Dekker, 1991.

    Google Scholar 

  66. W.J. Stewart. Introduction to the Numerical Solution of Markov Chains. Princeton University Press, 1994.

    Google Scholar 

  67. K.S. Trivedi. Probability and Statistics with Reliability, Queueing and Computer Science Applications. Prentice-Hall, 1982.

    Google Scholar 

  68. M. Veran and D. Potier. QNAP2: A portable environment for queueing system modeling. In D. Potier, editor, Modelling Techniques and Tools for Computer Performance Evaluation, pages 25–63._North-Holland, 1984.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Laboratory for Performance Evaluation and Distributed Systems, RWTH Aachen, 52056, Aachen, Germany

    Boudewijn R. Haverkort

Authors
  1. Boudewijn R. Haverkort
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Computer Science, Formal Methods and Tools Group, University of Twente, 217, 7500, AE, Enschede, The Netherlands

    Ed Brinksma , Holger Hermanns  & Joost-Pieter Katoen ,  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Haverkort, B.R. (2001). Markovian Models for Performance and Dependability Evaluation. In: Brinksma, E., Hermanns, H., Katoen, JP. (eds) Lectures on Formal Methods and PerformanceAnalysis. EEF School 2000. Lecture Notes in Computer Science, vol 2090. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44667-2_2

Download citation

  • DOI: https://doi.org/10.1007/3-540-44667-2_2

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-42479-6

  • Online ISBN: 978-3-540-44667-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation