control false discovery rate

Design, implement, or apply statistical procedures that control the expected proportion of false positives among declared discoveries across multiple hypothesis tests; this includes computing p-value thresholds and adjusted p-values using step-up/step-down algorithms (e.g., Benjamini–Hochberg and its variants) and producing or reporting set-level or group-level FDR estimates to make controlled selection decisions.

controlfalsediscoveryrate

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
-0.02
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$200K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

False Discovery Rate Adjustments for Average Significance Level Controlling Tests

Sep 27, 2022
TB
Timothy B. Armstrong
🏛️ University of Southern California

Classical false discovery rate (FDR) control methods, such as Benjamini–Hochberg (BH), rely on stringent pointwise control of Type I error (strong control), limiting their applicability under weaker inferential assumptions. This work addresses FDR control when only average-level (i.e., weak) control of significance level is required across tests. Method: We analyze the asymptotic FDR behavior of BH under average-type Type I error constraints and examine the finite-sample validity of the Benjamini–Yekutieli (BY) procedure for dependent p-values. Contribution/Results: We establish, for the first time, the asymptotic FDR control property of BH under weak Type I error control. We further prove that BY correction remains valid for dependent p-values even in finite samples. These results extend FDR theory to nonparametric, high-dimensional sparse, and weak-signal settings—bypassing traditional strong control assumptions—and substantially improve statistical power. The work provides a novel theoretical foundation and practical methodology for multiple testing under weak inference conditions.

Adjusting FDR for tests with average significance level controlEnabling FDR control in nonparametric and high-dimensional settingsExtending BH procedure to weakly dependent p-values asymptotically

This study addresses a critical limitation of traditional multiple testing procedures—such as the Benjamini–Hochberg (BH) method—which control the overall false discovery rate (FDR) but offer no guarantee regarding the reliability of boundary discoveries, i.e., the least significant rejections. The authors propose a novel two-stage adaptive approach: first estimating the number of true null hypotheses using non-significant test statistics, then applying an adjusted threshold within the Support Line (SL) framework to control the error probability of boundary discoveries. This work is the first to integrate adaptivity into boundary FDR control, providing rigorous error guarantees under independence and demonstrating robustness and enhanced power under positive dependence. Theoretical analysis confirms its validity, simulations show substantially improved statistical power over the original SL procedure, and real-world applicability is illustrated through a meta-analysis in psychology.

adaptive procedureboundary FDRerror control

Anytime-valid FDR control with the stopped e-BH procedure

Feb 12, 2025
HW
Hongjian Wang
🏛️ Carnegie Mellon University

This paper addresses the failure of the stopped e-BH (se-BH) procedure to control the false discovery rate (FDR) under arbitrary stopping times in dependent, data-streaming settings. We identify the root cause as incompatibility between local e-processes and the global filtration, leading to cross-stream information leakage. To resolve this, we introduce verifiable causal conditions—such as absence of unmeasured confounding—and rigorously prove that when e-processes across streams satisfy these conditions, they remain valid e-processes under the global filtration. Consequently, se-BH achieves anytime-valid FDR control for arbitrary stopping times. This is the first online FDR method that simultaneously provides theoretical guarantees and computational feasibility under non-i.i.d., sequentially arriving, and dependent data. The framework enables robust, adaptive multiple testing in real-time applications such as genomics.

Addresses information leakage in adaptively stopped e-processesControls FDR in sequential e-BH procedure with e-processesFormulates causal condition for valid FDR control

More powerful multiple testing under dependence via randomization

May 18, 2023
ZX
Ziyu Xu
🏛️ Carnegie Mellon University

This paper addresses the low statistical power of false discovery rate (FDR) and false coverage rate (FCR) control procedures under arbitrary dependence structures in multiple hypothesis testing. We propose a universal randomization-enhancement strategy based on a single uniform random variable. The method systematically boosts the power of classical procedures—including Benjamini–Yekutieli, e-BH, and Hommel—while preserving exact FDR/FCR control under arbitrary dependence. We provide the first rigorous proof that a single randomization step is never inferior to the original procedure and strictly improves power under any dependence structure; moreover, it unifies and strengthens diverse multiple testing procedures within the e-value framework. Theoretical analysis guarantees strict FDR/FCR control, and extensive simulations confirm substantial power gains. Our core innovation lies in achieving broad-spectrum power enhancement via an extremely simple randomization mechanism, thereby overcoming the long-standing power bottleneck of conventional methods under strong dependence.

Enhances FDR control procedures for dependent p-values and e-valuesImproves power of multiple testing under dependence via randomizationStrengthens Hommel test and post-selection inference for FCR control

E-values, Multiple Testing and Beyond

Dec 05, 2023
GL
Guanxun Li
🏛️ Beijing Normal University at Zhuhai | Texas A&M University

This paper addresses the underutilization of heterogeneity and structural information in multiple hypothesis testing by proposing a general e-value–based framework. Methodologically: (1) it introduces a data-dependent weighting scheme—including a leave-one-out heuristic—for flexible aggregation of e-values across subsets, test statistics, and structure-informed covariates; (2) it unifies and extends the Benjamini–Hochberg (BH) and Benjamini–Yekutieli (BY) procedures to accommodate mixed tests and joint group-level–global false discovery rate (FDR) control; (3) it develops a structure-adaptive e-BH procedure that relaxes the independence and homogeneity assumptions inherent in classical p-value–based methods. Theoretically, it guarantees strict finite-sample FDR control. Numerical experiments demonstrate substantial gains in statistical power over state-of-the-art baselines—particularly under heterogeneous, grouped, or covariate-structured settings.

Controlling false discovery rates in diverse testing scenariosDeveloping e-value aggregation methods for multiple testingIncorporating data-dependent weighting to enhance statistical power

Latest Papers

What's happening recently
View more

This work addresses a limitation of conventional approaches in multiple hypothesis testing within location families, which typically control the false discovery rate (FDR) only under the global null hypothesis. In practice, however, there is often a need to control FDR uniformly over non-significant regions across the entire parameter space. The paper reframes FDR as a function of the location parameter and proposes a natural extension of the Benjamini–Hochberg (BH) procedure that simultaneously controls the entire FDR curve without incurring additional computational cost. Theoretical analysis establishes that the proposed method guarantees the FDR curve remains below a user-specified level uniformly, and numerical experiments corroborate its effectiveness and practical utility.

Benjamini-Hochberg procedurefalse discovery rateFDR control

This work aims to enhance the statistical power of adaptive Benjamini–Hochberg (BH) procedures while maintaining control of the false discovery rate (FDR). By unifying existing adaptive FDR methods under a common framework—interpreting them as weighted BH procedures based on composite e-values (ep-BH)—the study reveals their shared structural foundation and demonstrates for the first time that most estimators of the proportion of true null hypotheses inherently correspond to composite e-values. Building on this insight, the authors propose a novel framework that uniformly improves upon nearly all existing methods without requiring additional assumptions, and they develop a new ep-BH procedure with finite-sample FDR guarantees. In canonical settings such as t-tests, the proposed method achieves consistent and robust power gains while rigorously controlling the FDR.

adaptive BH procedurecompound e-valuesfalse discovery rate

This study investigates the admissibility and complete class problems for false discovery rate (FDR) control procedures within the e-value framework. Drawing on statistical decision theory, it introduces strong and weak dominance relations to establish, for the first time, a theoretical foundation for admissibility in e-value-based multiple testing with FDR control. The main contributions include proving that every step-down procedure is strongly dominated by some weighted average eBH procedure; demonstrating that weighted average eBH procedures without constant terms are admissible at any FDR level; and showing that, under symmetry, this class of procedures forms a complete class, with its members being maximal only when the FDR threshold is sufficiently small—thereby establishing their structural optimality.

admissibilitycomplete classe-values

This work proposes a Posterior Conformal Selection (PH-CS) framework that overcomes the rigidity of traditional conformal selection methods, which require a pre-specified false discovery rate (FDR) threshold and thus struggle to balance selection size against FDR control. PH-CS eliminates the need for any preset FDR level by constructing a path of candidate selection sets and estimating their data-driven false discovery proportions (FDPs). Leveraging conformal e-values together with the e-BH procedure, the framework enables users to dynamically choose an optimal operating point based on a custom utility function. The method provides reliable average-case FDP estimates under finite samples, extends naturally to general risk control, and demonstrates competitive FDR control performance while accurately estimating FDP and satisfying utility constraints in both synthetic and real-data experiments.

conformal selectione-variablesfalse discovery rate

This study demonstrates that the Benjamini–Hochberg (BH) procedure may fail to control the nominal false discovery rate (FDR) in correlated two-sample Gaussian testing, thereby refuting the long-standing conjecture that BH maintains FDR control under dependence among Gaussian p-values. By constructing a factor model and combining interval arithmetic certification with rigorous mathematical analysis—supplemented by verification using GPT-5.6 Pro—the authors prove that, for sufficiently large numbers of hypotheses and a significance level α = 0.01, the actual FDR strictly exceeds 0.0104. These theoretical findings are corroborated by Monte Carlo simulations, providing the first conclusive evidence that the BH procedure can indeed lose FDR control under specific correlation structures.

Benjamini–Hochberg procedurecorrelated testsfalse discovery rate

Hot Scholars

OS

Osvaldo Simeone

King's College London
Information theorymachine learningquantum information processingwireless systems
AF

Amirmohammad Farzaneh

Research Associate, King's College London
Information TheoryConformal PredictionGraph TheoryCommunication Systems
AR

Aaditya Ramdas

Associate Professor (with tenure), Carnegie Mellon University
Machine LearningStatistics
MI

Michael I. Jordan

Professor of Electrical Engineering and Computer Sciences and Professor of Statistics, UC Berkeley
machine learningcomputer sciencestatisticsartificial intelligence
BJ

Bingyi Jing

Chair Professor, Southern University of Science & Technology
StatisticsData ScienceAI