A multiple-testing correction that controls the false discovery rate, the expected share of false positives among the results called significant. Sort the m p-values, find the largest rank k with p₍ₖ₎ ≤ (k / m) × q, and reject the k smallest.
Card 1 of 276. Answer: A multiple-testing correction that controls the false discovery rate, the expected share of false positives among the results called significant. Sort the m p-values, find the largest rank k with p₍ₖ₎ ≤ (k / m) × q, and reject the k smallest.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Corrections such as Bonferroni control the family-wise error rate, the chance of even one false positive, which becomes very strict when you run many tests. Benjamini-Hochberg instead accepts that a share q of the discoveries, often 5% or 10%, may be false, and in exchange finds far more real effects. Yoav Benjamini and Yosef Hochberg published it in 1995.
Example: ten tests give p-values of 0.001, 0.008, 0.012, 0.03, 0.041, 0.09, 0.2, 0.4, 0.6 and 0.9. With q = 0.05, the thresholds are 0.005, 0.010, 0.015, 0.020 and so on. The first three p-values sit under their thresholds and the fourth, 0.03, is above 0.020, as is every later one, so the procedure rejects three hypotheses. Bonferroni, which tests each against 0.05 / 10 = 0.005, rejects only one.
It is guaranteed when the tests are independent or positively correlated; the Benjamini-Yekutieli variant works under any dependence at the cost of power. It is standard in genomics, where thousands of genes are tested at once, and useful in A/B testing when an experiment tracks many metrics or segments.
