Abstract
This chapter briefly introduces the necessary concepts from the theory of stochastic processes (see for example Lamperti 1977; Doob 1953) that are needed for a proper definition of information rate and channel capacity, following Shannon.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
\(\mathbf{c} {=\log }_{2}10 +{ \frac{8} {10}\log }_{2}8 +{ \frac{2} {10}\log }_{2}2 {-\log }_{2}10 = 2.6\text{ bit}\).
- 2.
\(\text{ bit-rate} {=\log }_{2}10 + \frac{68} {100}\left ({\frac{16} {17}\log }_{2}\frac{16} {17} +{ \frac{1} {17}\log }_{2} \frac{1} {17}\right ) {=\log }_{2}10 + \frac{68} {100}\left (\frac{64} {17} {-\log }_{2}17\right ) {=\log }_{2}10\ -\ 0.322757 \cdot 0.68 = 3.10245\).
References
Ash, R. B. (1965). Information theory. New York, London, Sidney: Interscience.
Attneave, F. (1959). Applications of information theory to psychology. New York: Holt, Rinehart and Winston.
Billingsley, P. (1978). Ergodic theory and information. Huntington, NY: Robert E. Krieger Publishing Co.
Cover, T. M. & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.
Doob, J. L. (1953). Stochastic processes. New York: Wiley.
Friedman, N. A. (1970). Introduction to ergodic theory. New York: Van Nostrand Reinhold Company.
Gray, R. M. (1990). Entropy and information theory. New York: Springer.
Khinchin, A. (1957). Mathematical foundations of information theory. New York: Dover Publications, Inc.
Lamperti, J. (1966). Probability : A survey of the mathematical theory. Reading, Massachusetts: Benjamin/Cummings.
Lamperti, J. (1977). Stochastic processes - a survey of the mathematical theory. Applied Mathematical Sciences 23, New York: Springer.
Shannon, C. E. (1948). A mathematical theory of communication. Bell Systems Technical Journal, 27, 379–423, 623–656.
Topsøe, F. (1974). Informationstheorie: eine Einführung. Stuttgart: Teubner Verlag.
Walters, P. (1982). An introduction to ergodic theory. New York, Berlin, Heidelberg: Springer.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Palm, G. (2012). Stationary Processes and Their Information Rate. In: Novelty, Information and Surprise. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29075-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-29075-6_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29074-9
Online ISBN: 978-3-642-29075-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)