Asymptotic Equipartition Property

A theorem from information theory that is a simple consequence of the weak law of large numbers. It states that if a set of values , , ...,  is drawn independently from a random variable  distributed according to , then the joint probability  satisfies

where  is the entropy of the random variable .

REFERENCES:

Cover, T. M. and Thomas, J. A. Elements of Information Theory. New York: Wiley, 1991.