In statistics, the likelihood principle is the proposition that, given a statistical model, all the evidence in a sample relevant to model parameters is contained in the likelihood function.
The strong likelihood principle applies this same criterion to cases such as sequential experiments where the sample of data that is available results from applying a stopping rule to the observations earlier in the experiment.
lies not in the actual data collected, nor in the conduct of the experimenter, but in the two different designs of the experiment.
The use of frequentist methods involving p values leads to different inferences for the two cases above,[2] showing that the outcome of frequentist methods depends on the experimental procedure, and thus violates the likelihood principle.
The Neyman–Pearson lemma states the likelihood-ratio test is equally statistically powerful as the most powerful test for comparing two simple hypotheses at a given significance level, which gives a frequentist justification for the law of likelihood.
Some widely used methods of conventional statistics, for example many significance tests, are not consistent with the likelihood principle.
According to Giere (1977),[5] Birnbaum rejected[4] both his own conditionality principle and the likelihood principle because they were both incompatible with what he called the “confidence concept of statistical evidence”, which Birnbaum (1970) describes as taking “from the Neyman-Pearson approach techniques for systematically appraising and bounding the probabilities (under respective hypotheses) of seriously misleading interpretations of data” ([4] p. 1033).
Birnbaum later notes that it was the unqualified equivalence formulation of his 1962 version of the conditionality principle that led “to the monster of the likelihood axiom” ([6] p. 263).
Birnbaum's original argument for the likelihood principle has also been disputed by other statisticians including Akaike,[7] Evans[8] and philosophers of science, including Deborah Mayo.
[12] Unrealized events play a role in some common statistical methods.
For example, the result of a significance test depends on the p-value, the probability of a result as extreme or more extreme than the observation, and that probability may depend on the design of the experiment.
To the extent that the likelihood principle is accepted, such methods are therefore denied.
The following are a simple and more complicated example of those, using a commonly cited example called the optional stopping problem.
Suppose I tell you that I tossed a coin 12 times and in the process observed 3 heads.
Suppose a number of scientists are assessing the probability of a certain outcome (which we shall call 'success') in experimental trials.
Adam, a scientist, conducted 12 trials and obtains 3 successes and 9 failures.
He tested the null hypothesis that p, the success probability, is equal to a half, versus p < 0.5 .
Thus the null hypothesis is not rejected at the 5% significance level if we ignore the knowledge that the third success was the 12th result.
However observe that this first calculation also includes 12 token long sequences that end in tails contrary to the problem statement!
If we redo this calculation we realize the likelihood according to the null hypothesis must be the probability of a fair coin landing 2 or fewer heads on 11 trials multiplied with the probability of the fair coin landing a head for the 12th trial: which is 67/20481/2 = 67/4096 = 1.64% .
Charlotte, another scientist, reads Bill's paper and writes a letter, saying that it is possible that Adam kept trying until he obtained 3 successes, in which case the probability of needing to conduct 12 or more experiments is given by which is 134/40961/2 = 1.64% .
Results of this kind are considered by some as arguments against the likelihood principle.
For others it exemplifies the value of the likelihood principle and is an argument against significance tests.
Adam is very glad that he got his 3 successes after exactly 12 trials, and explains to his friend Charlotte that by coincidence he executed the second instruction.
Later, Adam is astonished to hear about Charlotte's letter, explaining that now the result is significant.