In statistics, the concept of being an invariant estimator is a criterion that can be used to compare the properties of different estimators for the same quantity.
It is a way of formalising the idea that an estimator should have certain intuitively appealing qualities.
Strictly speaking, "invariant" would mean that the estimates themselves are unchanged when both the measurements and the parameters are transformed in a compatible way, but the meaning has been extended to allow the estimates to change in appropriate ways with such transformations.
[1] The term equivariant estimator is used in formal mathematical contexts that include a precise description of the relation of the way the estimator changes in response to changes to the dataset and parameterisation: this corresponds to the use of "equivariance" in more general mathematics.
In statistical inference, there are several approaches to estimation theory that can be used to decide immediately what estimators should be used according to those approaches.
Similarly, the theory of classical statistical inference can sometimes lead to strong conclusions about what estimator should be used.
However, the usefulness of these theories depends on having a fully prescribed statistical model and may also depend on having a relevant loss function to determine the estimator.
Thus a Bayesian analysis might be undertaken, leading to a posterior distribution for relevant parameters, but the use of a specific utility or loss function may be unclear.
Ideas of invariance can then be applied to the task of summarising the posterior distribution.
In other cases, statistical analyses are undertaken without a fully defined statistical model or the classical theory of statistical inference cannot be readily applied because the family of models being considered are not amenable to such treatment.
The concept of invariance is sometimes used on its own as a way of choosing between estimators, but this is not necessarily definitive.
For example, a requirement of invariance may be incompatible with the requirement that the estimator be mean-unbiased; on the other hand, the criterion of median-unbiasedness is defined in terms of the estimator's sampling distribution and so is invariant under many transformations.
One use of the concept of invariance is where a class or family of estimators is proposed and a particular formulation must be selected amongst these.
One procedure is to impose relevant invariance properties and then to find the formulation within this class that has the best properties, leading to what is called the optimal invariant estimator.
There are several types of transformations that are usefully considered when dealing with invariant estimators.
Each gives rise to a class of estimators which are invariant to those particular types of transformation.
The combination of permutation invariance and location invariance for estimating a location parameter from an independent and identically distributed dataset using a weighted average implies that the weights should be identical and sum to one.
are modelled as a vector random variable having a probability density function
, is a function of the measurements and belongs to a set
The quality of the result is defined by a loss function
In statistical classification, the rule which assigns a class to a new data-item can be considered to be a special type of estimator.
A number of invariance-type considerations can be brought to bear in formulating prior knowledge for pattern recognition.
An invariant estimator is an estimator which obeys the following two rules:[citation needed] To define an invariant or equivariant estimator formally, some definitions related to groups of transformations are needed first.
Such an equivalence class is called an orbit (in
The invariant estimator in this case must satisfy thus it is of the form
The best invariant estimator is the one that brings the risk
, where θ is a parameter to be estimated, and where the loss function is
For the squared error loss case, the result is If
(i.e. a multivariate normal distribution with independent, unit-variance components) then If
(independent components having a Cauchy distribution with scale parameter σ) then