Empirical distribution function

In statistics, an empirical distribution function (commonly also called an empirical cumulative distribution function, eCDF) is the distribution function associated with the empirical measure of a sample.

[1] This cumulative distribution function is a step function that jumps up by 1/n at each of the n data points.

The empirical distribution function is an estimate of the cumulative distribution function that generated the points in the sample.

It converges with probability 1 to that underlying distribution, according to the Glivenko–Cantelli theorem.

A number of results exist to quantify the rate of convergence of the empirical distribution function to the underlying cumulative distribution function.

Let (X1, …, Xn) be independent, identically distributed real random variables with the common cumulative distribution function F(t).

Then the empirical distribution function is defined as[2] where

is a Bernoulli random variable with parameter p = F(t); hence

By the strong law of large numbers, the estimator

This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function.

There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over t:[5] The sup-norm in this expression is called the Kolmogorov–Smirnov statistic for testing the goodness-of-fit between the empirical distribution

The asymptotic distribution can be further characterized in several different ways.

First, the central limit theorem states that pointwise,

has asymptotically normal distribution with the standard

rate of convergence:[2] This result is extended by the Donsker’s theorem, which asserts that the empirical process

, converges in distribution in the Skorokhod space

[5] The covariance structure of this Gaussian process is The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding:[6] Alternatively, the rate of convergence of

can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression.

Number of results exist in this venue, for example the Dvoretzky–Kiefer–Wolfowitz inequality provides bound on the tail probabilities of

:[6] In fact, Kolmogorov has shown that if the cumulative distribution function F is continuous, then the expression

, which has the Kolmogorov distribution that does not depend on the form of F. Another result, which follows from the law of the iterated logarithm, is that [6] and As per Dvoretzky–Kiefer–Wolfowitz inequality the interval that contains the true CDF,

is specified as As per the above bounds, we can plot the Empirical CDF, CDF and confidence intervals for different distributions by using any one of the statistical implementations.

A non-exhaustive list of software implementations of Empirical Distribution function includes: