Abstract
A stochastic process or a random process is a family of random variables Xt,t ∈ I, taking values in some measurable space (E, ξ), with I = an interval of integers or reals. I has the interpretation of a (discrete or continuous) time interval, t thereby being a time index. The simplest random process in discrete time is a sequence of independent random variables. This can be considered as a process without memory: the value of the process at a given time instant is independent of the values it took in the past. The next level of complication one can conceive of is that of a one step memory: the value taken by the process at time n +1 depends on its past values only through its dependence, if any, on the value at time n, not otherwise. To be precise, it is conditionally independent of its values up to time n conditioned on its value at time n. This is called the Markov property and the process a Markov chain.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1995 Springer Science+Business Media New York
About this chapter
Cite this chapter
Borkar, V.S. (1995). Markov Chains. In: Probability Theory. Universitext. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-0791-7_5
Download citation
DOI: https://doi.org/10.1007/978-1-4612-0791-7_5
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-94558-3
Online ISBN: 978-1-4612-0791-7
eBook Packages: Springer Book Archive