Bootstrapping (statistics)
[1] Bootstrapping assigns measures of accuracy (bias, variance, confidence intervals, prediction error, etc.)
[16] Bootstrapping is also a convenient method that avoids the cost of repeating the experiment to get other groups of sample data.
[17] Although bootstrapping is (under some conditions) asymptotically consistent, it does not provide general finite-sample guarantees.
If the results may have substantial real-world consequences, then one should use as many samples as is reasonable, given available computing power and time.
Moreover, there is evidence that numbers of samples greater than 100 lead to negligible improvements in the estimation of standard errors.
[18] In fact, according to the original developer of the bootstrapping method, even setting the number of samples at 50 is likely to lead to fairly good standard error estimates.
As a result, confidence intervals on the basis of a Monte Carlo simulation of the bootstrap could be misleading.
Athreya states that "Unless one is reasonably sure that the underlying distribution is not heavy tailed, one should hesitate to use the naive bootstrap".
[2] The bootstrap is generally useful for estimating the distribution of a statistic (e.g. mean, variance) without using normality assumptions (as required, e.g., for a z-statistic or a t-statistic).
In particular, the bootstrap is useful when there is no analytical form or an asymptotic theory (e.g., an applicable central limit theorem) to help estimate the distribution of the statistics of interest.
The latter is a valid approximation in infinitely large samples due to the central limit theorem.
From this empirical distribution, one can derive a bootstrap confidence interval for the purpose of hypothesis testing.
For regression problems, as long as the data set is fairly large, this simple scheme is often acceptable.
[23] Under this scheme, a small amount of (usually normally distributed) zero-centered random noise is added onto each resampled observation.
is Based on the assumption that the original data set is a realization of a random sample from a distribution of a specific parametric type, in this case a parametric model is fitted by parameter θ, often by maximum likelihood, and samples of random numbers are drawn from this fitted model.
Although there are arguments in favor of using studentized residuals; in practice, it often makes little difference, and it is easy to compare the results of both schemes.
This method uses Gaussian process regression (GPR) to fit a probabilistic model from which replicates may then be drawn.
[26] Regression model: Gaussian process prior: For any finite collection of variables, x1, ..., xn, the function outputs
[26] The wild bootstrap, proposed originally by Wu (1986),[27] is suited when the model exhibits heteroskedasticity.
Vinod (2006),[31] presents a method that bootstraps time series data using maximum entropy principles satisfying the Ergodic theorem with mean-preserving and mass-preserving constraints.
[33] The bootstrap is a powerful technique although may require substantial computing resources in both time and memory.
The nonparametric bootstrap samples items from a list of size n with counts drawn from a multinomial distribution.
[35] A way to improve on the Poisson bootstrap, termed "sequential bootstrap", is by taking the first samples so that the proportion of unique values is ≈0.632 of the original sample size n. This provides a distribution with main empirical characteristics being within a distance of
There are several methods for constructing confidence intervals from the bootstrap distribution of a real parameter: Efron and Tibshirani[2] suggest the following algorithm for comparing the means of two independent samples: Let
However, note that whether the smoothed or standard bootstrap procedure is favorable is case-by-case and is shown to depend on both the underlying distribution function and on the quantity being estimated.
This paragraph summarizes more complete descriptions of stochastic convergence in van der Vaart and Wellner[50] and Kosorok.
[51] The bootstrap defines a stochastic process, a collection of random variables indexed by some set
[52] Horowitz goes on to recommend using a theorem from Mammen[53] that provides easier to check necessary and sufficient conditions for consistency for statistics of a certain common form.
This is adequate for most statistical applications since it implies confidence bands derived from the bootstrap are asymptotically valid.
[51] In simpler cases, it is possible to use the central limit theorem directly to show the consistency of the bootstrap procedure for estimating the distribution of the sample mean.
