Randomised decision rule

As nonrandomised alternatives always exist to randomised Bayes rules, randomisation is not needed in Bayesian statistics, although frequentist statistical theory sometimes requires the use of randomised rules to satisfy optimality conditions such as minimax, most notably when deriving confidence intervals and hypothesis tests about discrete probability distributions.

be a set of non-randomised decision rules with associated probabilities

[2] Alternatively, a randomised decision rule may assign probabilities directly on elements of the actions space

Under this approach, its loss function is also defined directly as:

[4] As with nonrandomised decision rules, randomised decision rules may satisfy favourable properties such as admissibility, minimaxity and Bayes.

This shall be illustrated in the case of a finite decision problem, i.e. a problem where the parameter space is a finite set of, say,

, is the set of all vectors in which each entry is the value of the risk function associated with a randomised decision rule under a certain parameter: it contains all vectors of the form

An admissible decision rule is one that is not dominated by any other decision rule, i.e. there is no decision rule that has equal risk as or lower risk than it for all parameters and strictly lower risk than it for some parameter.

In a finite decision problem, the risk point of an admissible decision rule has either lower x-coordinates or y-coordinates than all other risk points or, more formally, it is the set of rules with risk points of the form

[6][7] A minimax Bayes rule is one that minimises the supremum risk

[1] In a finite decision problem with two possible parameters, the minimax rule can be found by considering the family of squares

, then the admissible decision rule lying on the line is minimax.

holds for every point in the risk set, then the minimax rule can either be an extreme point (i.e. a nonrandomised decision rule) or a line connecting two extreme points (nonrandomised decision rules).

In the special case where the parameter space has two elements, the line

respectively, is a family of points with Bayes risk

The minimum Bayes risk for the decision problem is therefore the smallest

[10][11] This line may either touch only one extreme point of the risk set, i.e. correspond to a nonrandomised decision rule, or overlap with an entire side of the risk set, i.e. correspond to two nonrandomised decision rules and randomised decision rules combining the two.

This is illustrated by the three situations below: As different priors result in different slopes, the set of all rules that are Bayes with respect to some prior are the same as the set of admissible rules.

This is also true in the general case, even with infinite parameter space, infinite Bayes risk, and regardless of whether the infimum Bayes risk can be attained.

[3][12] This supports the intuitive notion that the statistician need not utilise randomisation to arrive at statistical decisions.

[4] As randomised Bayes rules always have nonrandomised alternatives, they are unnecessary in Bayesian statistics.

However, in frequentist statistics, randomised rules are theoretically necessary under certain situations,[13] and were thought to be useful in practice when they were first invented: Egon Pearson forecast that they 'will not meet with strong objection'.

[16] In the usual formulation of the likelihood ratio test, the null hypothesis is rejected whenever the likelihood ratio

This can be interpreted as flipping a biased coin with a probability

and rejecting the null hypothesis if a heads turns up.

, and that the test is unique under normal situations.

[19] As an example, consider the case where the underlying distribution is Bernoulli with probability

An analogous problem arises in the construction of confidence intervals.

For instance, the Clopper-Pearson interval is always conservative because of the discrete nature of the binomial distribution.

An alternative is to find the upper and lower confidence limits