Lady tasting tea
[1] The experiment is the original exposition of Fisher's notion of a null hypothesis, which is "never proved or established, but is possibly disproved, in the course of experimentation".
The woman in question, phycologist Muriel Bristol, claimed to be able to tell whether the tea or the milk was added first to a cup.
Her future husband, William Roach, suggested that Fisher give her eight cups, four of each variety, in random order.
[4] One could then ask what the probability was for her getting the specific number of cups she identified correct (in fact all eight), but just by chance.
Fisher's description is less than 10 pages in length and is notable for its simplicity and completeness regarding terminology, calculations and design of the experiment.
The test statistic is a simple count of the number of successful attempts to select the four cups prepared by a given method.
The frequencies of the possible numbers of successes, given in the final column of this table, are derived as follows.
For 0 successes, there is clearly only one set of four choices (namely, choosing all four incorrect cups) giving this result.
For one success and three failures, there are four correct cups of which one is selected, which by the combination formula can occur in
The frequencies of the other possible numbers of successes are calculated correspondingly.
The critical region for rejection of the null of no ability to distinguish was the single case of 4 successes of 4 possible, based on the conventional probability criterion < 5%.
Thus, if and only if the lady properly categorized all 8 cups was Fisher willing to reject the null hypothesis – effectively acknowledging the lady's ability at a 1.4% significance level (but without quantifying her ability).
Fisher later discussed the benefits of more trials and repeated tests.
David Salsburg reports that a colleague of Fisher, H. Fairfield Smith, revealed that in the actual experiment the lady succeeded in identifying all eight cups correctly.
David Salsburg published a popular science book entitled The Lady Tasting Tea,[6] which describes Fisher's experiment and ideas on randomization.
Deb Basu wrote that "the famous case of the 'lady tasting tea'" was "one of the two supporting pillars ... of the randomization analysis of experimental data.
