De Finetti's theorem

In probability theory, de Finetti's theorem states that exchangeable observations are conditionally independent relative to some latent variable.

It is named in honor of Bruno de Finetti, and one of its uses is in providing a pragmatic approach to de Finetti's well-known dictum "Probability does not exist".

A Bayesian statistician often seeks the conditional probability distribution of a random quantity given the data.

De Finetti's theorem explains a mathematical relationship between independence and exchangeability.

If an identically distributed sequence is independent, then the sequence is exchangeable; however, the converse is false—there exist exchangeable random variables that are not statistically independent, for example the Pólya urn model.

De Finetti's theorem states that the probability distribution of any infinite exchangeable sequence of Bernoulli random variables is a "mixture" of the probability distributions of independent and identically distributed sequences of Bernoulli random variables.

"Mixture", in this sense, means a weighted average, but this need not mean a finite or countably infinite (i.e., discrete) weighted average: it can be an integral over a measure rather than a sum.

More precisely, suppose X1, X2, X3, ... is an infinite exchangeable sequence of Bernoulli-distributed random variables.

Then there is some probability measure m on the interval [0, 1] and some random variable Y such that Suppose

is an infinite exchangeable sequence of Bernoulli random variables.

According to David Spiegelhalter (ref 1) the theorem provides a pragmatic approach to de Finetti's statement that "Probability does not exist".

If our view of the probability of a sequence of events is subjective but remains unaffected by the order in which we make our observations, then the sequence can be regarded as exchangeable.

De Finetti's theorem then implies that believing the sequence to be exchangeable is mathematically equivalent to acting as if the events are independent and have an objective underlying probability of occurring, with our uncertainty about what that probability is being expressed by a subjective probability distribution function.

According to Spiegelhalter: ″This is remarkable: it shows that, starting from a specific, but purely subjective, expression of convictions, we should act as if events were driven by objective chances."

As a concrete example, we construct a sequence of random variables, by "mixing" two i.i.d.

In view of the strong law of large numbers, we can say that Rather than concentrating probability 1/2 at each of two points between 0 and 1, the "mixing distribution" can be any probability distribution supported on the interval from 0 to 1; which one it is depends on the joint distribution of the infinite sequence of Bernoulli random variables.

The definition of exchangeability, and the statement of the theorem, also makes sense for finite length sequences but the theorem is not generally true in that case.

The simplest example of an exchangeable sequence of Bernoulli random variables that cannot be so extended is the one in which X1 = 1 − X2 and X1 is either 0 or 1, each with probability 1/2.

De Finetti's theorem can be expressed as a categorical limit in the category of Markov kernels.

In terms of category theory, we have a diagram with a single object,

is equivalently a Markov kernel from the one-point measurable space.

De Finetti's theorem can be now stated as the fact that the space

(Giry monad) forms a universal (or limit) cone.

constructed as follows, using the Kolmogorov extension theorem: for all measurable subsets

Note that we can interpret this kernel as taking a probability measure

, i.e. making the following diagram commute: In particular, for any exchangeable probability measure

Versions of de Finetti's theorem for finite exchangeable sequences,[6][7] and for Markov exchangeable sequences[8] have been proved by Diaconis and Freedman and by Kerns and Szekely.

Two notions of partial exchangeability of arrays, known as separate and joint exchangeability lead to extensions of de Finetti's theorem for arrays by Aldous and Hoover.

[9] The computable de Finetti theorem shows that if an exchangeable sequence of real random variables is given by a computer program, then a program which samples from the mixing measure can be automatically recovered.

[10] In the setting of free probability, there is a noncommutative extension of de Finetti's theorem which characterizes noncommutative sequences invariant under quantum permutations.