A unified formalization of large deviation theory was developed in 1966, in a paper by Varadhan.
[1] Large deviations theory formalizes the heuristic ideas of concentration of measures and widely generalizes the notion of convergence of probability measures.
Roughly speaking, large deviations theory concerns itself with the exponential decline of the probability measures of certain kinds of extreme or tail events.
Any large deviation is done in the least unlikely of all the unlikely ways!Consider a sequence of independent tosses of a fair coin.
The central limit theorem can provide more detailed information about the behavior of
However, the large deviation theory can provide answers for such problems.
; that it's appropriate for coin tosses follows from the asymptotic equipartition property applied to a Bernoulli trial.
cannot be replaced with a larger number which would yield a strict inequality for all positive
[3] (However, the exponential bound can still be reduced by a subexponential factor on the order of
; this follows from the Stirling approximation applied to the binomial coefficient appearing in the Bernoulli distribution.)
This formula approximates any tail probability of the sample mean of i.i.d.
variables and gives its convergence as the number of samples increases.
In the above example of coin-tossing we explicitly assumed that each toss is an independent trial, and the probability of getting head or tail is always the same.
random variables whose common distribution satisfies a certain growth condition.
This is given by a Legendre–Fenchel transformation,[6] where is called the cumulant generating function (CGF) and
is an irreducible and aperiodic Markov chain, the variant of the basic large deviations result stated above may hold.
[citation needed] The previous example controlled the probability of the event
[citation needed] The first rigorous results concerning large deviations are due to the Swedish mathematician Harald Cramér, who applied them to model the insurance business.
random variables, where the rate function is expressed as a power series.
A very incomplete list of mathematicians who have made important advances would include Petrov,[10] Sanov,[11] S.R.S.
Varadhan (who has won the Abel prize for his contribution to the theory), D. Ruelle, O.E.
Lanford, Mark Freidlin, Alexander D. Wentzell, Amir Dembo, and Ofer Zeitouni.
[12] Principles of large deviations may be effectively applied to gather information out of a probabilistic model.
In physics, the best known application of large deviations theory arise in thermodynamics and statistical mechanics (in connection with relating entropy with rate function).
The rate function is related to the entropy in statistical mechanics.
And in most practical situations we shall indeed obtain this macro-state for large numbers of trials.
The "rate function" on the other hand measures the probability of appearance of a particular macro-state.
The smaller the rate function the higher is the chance of a macro-state appearing.
There is a relation between the "rate function" in large deviations theory and the Kullback–Leibler divergence, the connection is established by Sanov's theorem (see Sanov[11] and Novak,[13] ch.
In a special case, large deviations are closely related to the concept of Gromov–Hausdorff limits.