In statistics, the Holm–Bonferroni method, also called the Holm method or Bonferroni–Holm method, is used to counteract the problem of multiple comparisons. It is intended to control the family-wise error rate (FWER) and offers a simple test uniformly more powerful than the Bonferroni correction. It is named after Sture Holm, who codified the method, and Carlo Emilio Bonferroni.
Motivation When considering several hypotheses, the problem of multiplicity arises: the more hypotheses are tested, the higher the probability of obtaining Type I errors (false positives). The Holm–Bonferroni method is one of many approaches for controlling the FWER, i.e., the probability that one or more Type I errors will occur, by adjusting the rejection criterion for each of the individual hypotheses.
Formulation The method is as follows:
Suppose you have m {\displaystyle m} p-values, sorted into order lowest-to-highest P 1 , … , P m {\displaystyle P_{1},\ldots ,P_{m}} , for the corresponding null hypotheses H 1 , … , H m {\displaystyle H_{1},\ldots ,H_{m}} . And suppose the desired maximum FWER is α {\displaystyle \alpha } . Is P 1 ≤ α / m {\displaystyle P_{1}\leq \alpha /m} ? If so, reject H 1 {\displaystyle H_{1}} and continue to the next step, otherwise EXIT. Is P 2 ≤ α / ( m − 1 ) {\displaystyle P_{2}\leq \alpha /(m-1)} ? If so, reject H 2 {\displaystyle H_{2}} also, and continue to the next step, otherwise EXIT. And so on: for each P value, test whether P k ≤ α m + 1 − k {\displaystyle P_{k}\leq {\frac {\alpha }{m+1-k}}} . If so, reject H k {\displaystyle H_{k}} and continue to examine the larger P values, otherwise EXIT. This method ensures that the FWER is at most α {\displaystyle \alpha } , in the strong sense.
Rationale The simple Bonferroni correction rejects only null hypotheses with p-value less than or equal to α m {\displaystyle {\frac {\alpha }{m}}} , in order to ensure that the FWER, i.e., the risk of rejecting one or more true null hypotheses (i.e., of committing one or more type I errors) is at most α {\displaystyle \alpha } . The cost of this protection against type I errors is an increased risk of failing to reject one or more false null hypotheses (i.e., of committing one or more type II errors). The Holm–Bonferroni method also controls the FWER at α {\displaystyle \alpha } , but with a lower increase of type II error risk than the classical Bonferroni method. The Holm–Bonferroni method sorts the p-values from lowest to highest and compares them to nominal alpha levels of α m {\displaystyle {\frac {\alpha }{m}}} to α {\displaystyle \alpha } (respectively), namely the values α m , α m − 1 , … , α 2 , α 1 {\displaystyle {\frac {\alpha }{m}},{\frac {\alpha }{m-1}},\ldots ,{\frac {\alpha }{2}},{\frac {\alpha }{1}}} .
The index k {\displaystyle k} identifies the first p-value that is not low enough to validate rejection. Therefore, the null hypotheses H ( 1 ) , … , H ( k − 1 ) {\displaystyle H_{(1)},\ldots ,H_{(k-1)}} are rejected, while the null hypotheses H ( k ) , . . . , H ( m ) {\displaystyle H_{(k)},...,H_{(m)}} are not rejected. If k = 1 {\displaystyle k=1} then no p-values were low enough for rejection, therefore no null hypotheses are rejected. If no such index k {\displaystyle k} could be found then all p-values were low enough for rejection, therefore all null hypotheses are rejected (none are accepted).
… excerpt ends here. Continue reading the full article.
