It is intended to control the family-wise error rate (FWER) and offers a simple test uniformly more powerful than the Bonferroni correction.
It is named after Sture Holm, who codified the method, and Carlo Emilio Bonferroni.
When considering several hypotheses, the problem of multiplicity arises: the more hypotheses are tested, the higher the probability of obtaining Type I errors (false positives).
The Holm–Bonferroni method is one of many approaches for controlling the FWER, i.e., the probability that one or more Type I errors will occur, by adjusting the rejection criterion for each of the individual hypotheses.
The simple Bonferroni correction rejects only null hypotheses with p-value less than or equal to
, in order to ensure that the FWER, i.e., the risk of rejecting one or more true null hypotheses (i.e., of committing one or more type I errors) is at most
The cost of this protection against type I errors is an increased risk of failing to reject one or more false null hypotheses (i.e., of committing one or more type II errors).
, but with a lower increase of type II error risk than the classical Bonferroni method.
The Holm–Bonferroni method sorts the p-values from lowest to highest and compares them to nominal alpha levels of
be the set of indices corresponding to the (unknown) true null hypotheses, having
First note that, in this case, there is at least one true hypothesis, so
Subadditivity of the probability measure implies that
The Holm–Bonferroni method can be viewed as a closed testing procedure,[2] with the Bonferroni correction applied locally on each of the intersections of null hypotheses.
The closure principle states that a hypothesis
or less comparisons, while the number of all intersections of null hypotheses to be tested is of order
It controls the FWER in the strong sense.
is not rejected too, such that there exists at least one intersection hypothesis for each of elementary hypotheses
are not rejected while controlling the family-wise error rate at level
This is because the testing procedure stops once a failure to reject occurs.
When the hypothesis tests are not negatively dependent, it is possible to replace
with: resulting in a slightly more powerful test.
Similar adjusted p-values for Holm-Šidák method can be defined recursively as
The weighted adjusted p-values are:[citation needed] A hypothesis is rejected at level α if and only if its adjusted p-value is less than α.
In the earlier example using equal weights, the adjusted p-values are 0.03, 0.06, 0.06, and 0.02.
This is another way to see that using α = 0.05, only hypotheses one and four are rejected by this procedure.
is made after finding the maximal index
However, the Hochberg procedure requires the hypotheses to be independent or under certain forms of positive dependence, whereas Holm–Bonferroni can be applied without such assumptions.
[3] Carlo Emilio Bonferroni did not take part in inventing the method described here.
Holm originally called the method the "sequentially rejective Bonferroni test", and it became known as Holm–Bonferroni only after some time.
Holm's motives for naming his method after Bonferroni are explained in the original paper: "The use of the Boole inequality within multiple inference theory is usually called the Bonferroni technique, and for this reason we will call our test the sequentially rejective Bonferroni test."