This concept is analogous to Pareto efficiency.
, which specifies the loss we would incur by taking action
Usually we will take this action after observing data
(It is possible though unconventional to recast the following definitions in terms of a utility function, which is the negative of the loss.)
Define the risk function as the expectation Whether a decision rule
has low risk depends on the true state of nature
Thus an admissible decision rule is a maximal element with respect to the above partial order.
An inadmissible rule is not preferred (except for reasons of simplicity or computational efficiency), since by definition there is some other rule that will achieve equal or lower risk for all
is admissible does not mean it is a good rule to use.
Being admissible means there is no other single rule that is always as good or better – but other admissible rules might achieve lower risk for most
(The Bayes risk discussed below is a way of explicitly considering which
be a probability distribution on the states of nature.
From a Bayesian point of view, we would regard it as a prior distribution.
That is, it is our believed probability distribution on the states of nature, prior to observing data.
In the Bayesian approach to decision theory, the observed
Whereas the frequentist approach (i.e., risk) averages over possible samples
, the Bayesian would fix the observed sample
Thus, the Bayesian approach is to consider for our observed
There may be more than one generalized Bayes rule, since there may be multiple choices of
At first, this may appear rather different from the Bayes rule approach of the previous section, not a generalization.
in Bayesian fashion, and the Bayes risk may be recovered as the expectation over
Then why is the notion of generalized Bayes rule an improvement?
In this case it is still useful to define a generalized Bayes rule
In addition, a generalized Bayes rule may be desirable because it must choose a minimum-expected-loss action
, whereas a Bayes rule would be allowed to deviate from this policy on a set
According to the complete class theorems, under mild conditions every admissible rule is a (generalized) Bayes rule (with respect to some prior
Thus, in frequentist decision theory it is sufficient to consider only (generalized) Bayes rules.
Conversely, while Bayes rules with respect to proper priors are virtually always admissible, generalized Bayes rules corresponding to improper priors need not yield admissible procedures.
The James–Stein estimator is a nonlinear estimator of the mean of Gaussian random vectors and can be shown to dominate the ordinary least squares technique with respect to a mean-squared-error loss function.
Some others of the standard estimates associated with the normal distribution are also inadmissible: for example, the sample estimate of the variance when the population mean and variance are unknown.