False discovery rate
In statistics, the false discovery rate (FDR) is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons.
FDR-controlling procedures are designed to control the FDR, which is the expected proportion of "discoveries" (rejected null hypotheses) that are false (incorrect rejections of the null).
[1] Equivalently, the FDR is the expected ratio of the number of false positive classifications (false discoveries) to the total number of positive classifications (rejections of the null).
Thus, FDR-controlling procedures have greater power, at the cost of increased numbers of Type I errors.
[2] The modern widespread use of the FDR is believed to stem from, and be motivated by, the development in technologies that allowed the collection and analysis of a large number of distinct variables in several individuals (e.g., the expression level of each of 10,000 different genes in 100 different persons).
[3] By the late 1980s and 1990s, the development of "high-throughput" sciences, such as genomics, allowed for rapid data acquisition.
This, coupled with the growth in computing power, made it possible to seamlessly perform a very high number of statistical tests on a given data set.
The technology of microarrays was a prototypical example, as it enabled thousands of genes to be tested simultaneously for differential expression between two biological conditions.
[4] As high-throughput technologies became common, technological and/or financial constraints led researchers to collect datasets with relatively small sample sizes (e.g. few individuals being tested) and large numbers of variables being measured per sample (e.g. thousands of gene expression levels).
This created a need within many scientific communities to abandon FWER and unadjusted multiple hypothesis testing for other ways to highlight and rank in publications those variables showing marked effects across individuals or treatments that would otherwise be dismissed as non-significant after standard correction for multiple tests.
In response to this, a variety of error rates have been proposed—and become commonly used in publications—that are less conservative than FWER in flagging possibly noteworthy observations.
The FDR is useful when researchers are looking for "discoveries" that will give them followup work (E.g.: detecting promising genes for followup studies), and are interested in controlling the proportion of "false leads" they are willing to accept.
The FDR concept was formally described by Yoav Benjamini and Yosef Hochberg in 1995[1] (BH procedure) as a less conservative and arguably more appropriate approach for identifying the important few from the trivial many effects tested.
The FDR has been particularly influential, as it was the first alternative to the FWER to gain broad acceptance in many scientific fields (especially in the life sciences, from genetics to biochemistry, oncology and plant sciences).
[5] Prior to the 1995 introduction of the FDR concept, various precursor ideas had been considered in the statistics literature.
[1] In 1986, R. J. Simes offered the same procedure as the "Simes procedure", in order to control the FWER in the weak sense (under the intersection null hypothesis) when the statistics are independent.
[1] The following table defines the possible outcomes when testing multiple null hypotheses.
Summing each type of outcome over all Hi yields the following random variables: In m hypothesis tests of which
In a similar way, in a "step-down" procedure we move from a large corresponding test statistic to a smaller one.
is inserted into the BH procedure, it is no longer guaranteed to achieve FDR control at the desired level.
The Benjamini–Yekutieli procedure controls the false discovery rate under arbitrary dependence assumptions.
Using a multiplicity procedure that controls the FDR criterion is adaptive and scalable.
Meaning that controlling the FDR can be very permissive (if the data justify it), or conservative (acting close to control of FWER for sparse problem) - all depending on the number of hypotheses tested and the level of significance.
For example: Controlling the FDR using the linear step-up BH procedure, at level q, has several properties related to the dependency structure between the test statistics of the m null hypotheses that are being corrected for.
The average power of the Benjamini-Hochberg procedure can be computed analytically[18] The discovery of the FDR was preceded and followed by many other types of error rates.
These include: The false coverage rate (FCR) is, in a sense, the FDR analog to the confidence interval.
FCR indicates the average rate of false coverage, namely, not covering the true parameters, among the selected intervals.
Intervals with simultaneous coverage probability 1−q can control the FCR to be bounded by q.
There are many FCR procedures such as: Bonferroni-Selected–Bonferroni-Adjusted,[citation needed] Adjusted BH-Selected CIs (Benjamini and Yekutieli (2005)),[24] Bayes FCR (Yekutieli (2008)),[citation needed] and other Bayes methods.
[25] Connections have been made between the FDR and Bayesian approaches (including empirical Bayes methods),[21][26][27] thresholding wavelets coefficients and model selection,[28][29][30][31][32] and generalizing the confidence interval into the false coverage statement rate (FCR).
