Abstract
Consider a random experiment whose possible outcomes arez 1,z 2,...,z n. Let the prior probabilities be p1 0, ...,pn 0, and let the posterior probabilities bep 1,...,p n. It is shown that, subject to certain prescribed and intuitively reasonable conditions, the expressionI =k ⌆ p i In (p i/p i 0), wherek is a positive constant, is the unique expression for the information contained in a message which alters the probabilities from thep i 0 to thep i.
Similar content being viewed by others
References
C.E. Shannon,Bell System Tech. J. 27, 379 and 623 (1948). Reprinted in C. E. Shannon and W. Weaver,The Mathematical Theory of Communication (University of Illinois Press, Urbana, Illinois, 1949),
A. Feinstein,Foundations of Information Theory (McGraw-Hill Book Company, Inc., New York, 1958).
A. I. Khinchin,Mathematical Foundations of Information Theory (Dover Publications, Inc., New York, 1957).
F. M. Reza,An Introduction to Information Theory (McGraw-Hill Book Company, Inc., New York, 1961).
A. Katz,Principles of Statistical Mechanics: The Information Theory Approach (W. H. Freeman and Company, San Francisco, 1967).
S. Kullback,Information Theory and Statistics (John Wiley & Sons, Inc., New York, 1959).
A. Renyi,Wahrscheinlichkeitrechnung (VEB Deutscher Verlag der Wissenschaften, Berlin, 1966).
M. S. Pinsker,Information and Information Stability of Random Variables and Processes, translated and edited by A. Feinstein (Holden-Day, Inc., San Francisco, 1964).
F. Schlögl, inStatistical Mechanics (Proc. of the IUPAP Meeting, Copenhagen, 1966), edited by T. A. Bak (W. A. Benjamin, Inc., New York, 1967); inProc. of the International Conf. on Statistical Mechanics (Kyoto, 1968) (supplement toJ. Phys. Soc. Japan 26, 1969).
A. Hobson,Concepts in Statistical Mechanics (Gordon and Breach, Science Publishers, Inc., 1970).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hobson, A. A new theorem of information theory. J Stat Phys 1, 383–391 (1969). https://doi.org/10.1007/BF01106578
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF01106578