Duncan's MRT belongs to the general class of multiple comparison procedures that use the studentized range statistic qr to compare sets of means.
David B. Duncan developed this test as a modification of the Student–Newman–Keuls method that would have greater power.
Duncan's MRT is especially protective against false negative (Type II) error at the expense of having a greater risk of making false positive (Type I) errors.
, which is independent of the observed means and is based on a number of degrees of freedom, denoted by
An algorithm for performing the test is as follows: Duncan's multiple range test makes use of the studentized range distribution in order to determine critical values for comparisons between means.
Note that different comparisons between means may differ by their significance levels- since the significance level is subject to the size of the subset of means in question.
degrees of freedom for the second sample (see studentized range for more information).
A word of caution is needed here: notations for Q and R are not the same throughout literature, where Q is sometimes denoted as the shortest significant interval, and R as the significant quantile for studentized range distribution (Duncan's 1955 paper uses both notations in different parts).
Then, the observed differences between means are tested, beginning with the largest versus smallest, which would be compared with the least significant range
The new multiple range test proposed by Duncan makes use of special protection levels based upon degrees of freedom.
that is, the probability that one finds no significant differences in making p-1 independent tests, each at protection level
, the protection level can be tabulated for various value of r as follows: Note that although this procedure makes use of the Studentized range, his error rate is neither on an experiment-wise basis (as with Tukey's) nor on a per- comparisons basis.
Duncan's multiple range test does not control the family-wise error rate.
Duncan's Bayesian MCP discusses the differences between ordered group means, where the statistics in question are pairwise comparison (no equivalent is defined for the property of a subset having 'significantly different' property).
Duncan modeled the consequences of two or more means being equal using additive loss functions within and across the pairwise comparisons.
If one assumes the same loss function across the pairwise comparisons, one needs to specify only one constant K, and this indicates the relative seriousness of type I to type II errors in each pairwise comparison.
A study, which performed by Juliet Popper Shaffer (1998), has shown that the method proposed by Duncan, modified to provide weak control of FWE and using an empirical estimate of the variance of the population means, has good properties both from the Bayesian point of view, as a minimum- risk method, and from the frequentist point of view, with good average power.
In addition, results indicate considerable similarity in both risk and average power between Duncan's modified procedure and the Benjamini and Hochberg (1995) False discovery rate -controlling procedure, with the same weak family-wise error control.
Duncan's test has been criticised as being too liberal by many statisticians including Henry Scheffé, and John W. Tukey.
Duncan argued that a more liberal procedure was appropriate because in real world practice the global null hypothesis H0 = "All means are equal" is often false and thus traditional statisticians overprotect a probably false null hypothesis against type I errors.
According to Duncan, one should adjust the protection levels for different p-mean comparisons according to the problem discussed.
Duncan's multiple range test is very “liberal” in terms of Type I errors.
The following example will illustrate why: Let us assume one is truly interested, as Duncan suggested, only with the correct ranking of subsets of size 4 or below.
Let us also assume that one performs the simple pairwise comparison with a protection level
As we can see, the test has two main problems, regarding the type I errors: Therefore, it is advised not to use the procedure discussed.
Duncan later developed the Duncan–Waller test which is based on Bayesian principles.
It uses the obtained value of F to estimate the prior probability of the null hypothesis being true.
If one still wishes to address the problem of finding similar subsets of group means, other solutions are found in literature.
Tukey's range test is commonly used to compare pairs of means, this procedure controls the family-wise error rate in the strong sense.
Another solution is to perform Student's t-test of all pairs of means, and then to use FDR Controlling procedure (to control the expected proportion of incorrectly rejected null hypotheses).