It is applicable when the random variable being considered can be defined as a differentiable function of a random variable which is asymptotically Gaussian.
The delta method was derived from propagation of error, and the idea behind was known in the early 20th century.
[1] Its statistical application can be traced as far back as 1928 by T. L.
[2] A formal description of the method was presented by J. L. Doob in 1935.
[4] While the delta method generalizes easily to a multivariate setting, careful motivation of the technique is more easily demonstrated in univariate terms.
Roughly, if there is a sequence of random variables Xn satisfying where θ and σ2 are finite valued constants and
denotes convergence in distribution, then for any function g satisfying the property that its first derivative, evaluated at
When g is applied to a random variable such as the mean, the delta method would tend to work better as the sample size increases, since it would help reduce the variance, and thus the taylor approximation would be applied to a smaller range of the function g at the point of interest.
Demonstration of this result is fairly straightforward under the assumption that
To begin, we use the mean value theorem (i.e.: the first order approximation of a Taylor series using Taylor's theorem): where
gives Since by assumption, it follows immediately from appeal to Slutsky's theorem that This concludes the proof.
Alternatively, one can add one more step at the end, to obtain the order of approximation: This suggests that the error in the approximation converges to 0 in probability.
By definition, a consistent estimator B converges in probability to its true value β, and often a central limit theorem can be applied to obtain asymptotic normality: where n is the number of observations and Σ is a (symmetric positive semi-definite) covariance matrix.
Keeping only the first two terms of the Taylor series, and using vector notation for the gradient, we can estimate h(B) as which implies the variance of h(B) is approximately One can use the mean value theorem (for real-valued functions of many variables) to see that this does not rely on taking first order approximation.
The delta method therefore implies that or in univariate terms, Suppose Xn is binomial with parameters
does not actually exist (since Xn can be zero), the asymptotic variance of
are estimates of different group rates from independent samples of sizes n and m respectively, then the logarithm of the estimated relative risk
has asymptotic variance equal to This is useful to construct a hypothesis test or to make a confidence interval for the relative risk.
The delta method is often used in a form that is essentially identical to that above, but without the assumption that Xn or B is asymptotically normal.
The results then just give approximations to the means and covariances of the transformed quantities.
For example, the formulae presented in Klein (1953, p. 258) are:[5] where hr is the rth element of h(B) and Bi is the ith element of B.
However, if g′′(θ) exists and is not zero, the second-order delta method can be applied.
The second-order delta method is also useful in conducting a more accurate approximation of
's distribution when sample size is small.
can be approximated as the weighted sum of a standard normal and a chi-square with degree-of-freedom of 1.
A version of the delta method exists in nonparametric statistics.
be an independent and identically distributed random variable with a sample of size
is Hadamard differentiable with respect to the Chebyshev metric, then where
denoting the empirical influence function for
pointwise asymptotic confidence interval for