benjamini-hochberg procedure

Classical false discovery rate (FDR) control methods, such as Benjamini–Hochberg (BH), rely on stringent pointwise control of Type I error (strong control), limiting their applicability under weaker inferential assumptions. This work addresses FDR control when only average-level (i.e., weak) control of significance level is required across tests. Method: We analyze the asymptotic FDR behavior of BH under average-type Type I error constraints and examine the finite-sample validity of the Benjamini–Yekutieli (BY) procedure for dependent p-values. Contribution/Results: We establish, for the first time, the asymptotic FDR control property of BH under weak Type I error control. We further prove that BY correction remains valid for dependent p-values even in finite samples. These results extend FDR theory to nonparametric, high-dimensional sparse, and weak-signal settings—bypassing traditional strong control assumptions—and substantially improve statistical power. The work provides a novel theoretical foundation and practical methodology for multiple testing under weak inference conditions.

This paper investigates the false discovery rate (FDR) control performance of the Benjamini–Hochberg (BH) procedure under *composite p-values*—i.e., p-values that are only required to be super-uniform on all true null hypotheses, a weaker condition than standard uniformity. Theoretical analysis establishes that, under independence, FDR ≤ 1.93α; when all nulls are true, FDR ≤ α + 2α²; and under positive dependence, FDR may inflate by a factor of O(log m), with matching tight upper and lower bound constructions. This work provides the first systematic characterization of the robustness boundary of the BH procedure to composite p-values, precisely delineating its FDR control capability—and inherent limitations—when the classical uniformity assumption is relaxed. The results establish a new theoretical foundation for multiple testing in high-dimensional settings with complex dependencies, enhancing both flexibility and statistical power.

In multiple testing, the p-value lacks alternative information, and inaccurate local false discovery rate (Lfdr) estimation hinders simultaneous control of the false discovery rate (FDR) and statistical power. This paper proposes the *rho-value* framework: first constructing an optimal ranking using structured test statistics and auxiliary covariates, then applying a p-value–style thresholding rule to guarantee strict FDR control. The framework unifies the p-value and Lfdr paradigms for the first time; rho-values provide rigorous FDR control with asymptotic optimality, without requiring consistent Lfdr estimation and exhibiting robustness to auxiliary covariates. Simulation and real-data analyses demonstrate that, under stringent FDR control, rho-values improve statistical power by 12%–28% over BH, AdaPT, and IHW—particularly in sparse, heterogeneous signal settings.

This paper addresses the underutilization of heterogeneity and structural information in multiple hypothesis testing by proposing a general e-value–based framework. Methodologically: (1) it introduces a data-dependent weighting scheme—including a leave-one-out heuristic—for flexible aggregation of e-values across subsets, test statistics, and structure-informed covariates; (2) it unifies and extends the Benjamini–Hochberg (BH) and Benjamini–Yekutieli (BY) procedures to accommodate mixed tests and joint group-level–global false discovery rate (FDR) control; (3) it develops a structure-adaptive e-BH procedure that relaxes the independence and homogeneity assumptions inherent in classical p-value–based methods. Theoretically, it guarantees strict finite-sample FDR control. Numerical experiments demonstrate substantial gains in statistical power over state-of-the-art baselines—particularly under heterogeneous, grouped, or covariate-structured settings.

In multiple hypothesis testing scenarios where p-values or test statistics are insufficient, conventional methods suffer from low statistical power and lack rigorous finite-sample false discovery rate (FDR) or false discovery proportion (FDP) control. Method: This paper proposes RESET—a novel framework—and its ensemble extension, RESET Ensemble, which for the first time integrates semi-supervised learning with strict finite-sample FDR/FDP control. Leveraging an innovative data-splitting protocol, RESET effectively incorporates side information while preserving theoretical guarantees—without manual hyperparameter tuning or model selection. It is compatible with both p-value–based and competition-based testing paradigms. Contribution/Results: RESET achieves significant power gains over existing methods while maintaining stringent FDR/FDP control under finite samples. It is computationally efficient, theoretically rigorous, and—uniquely among general-purpose multiple testing frameworks—supports user-selectable, provably valid control of either FDR or FDP in finite samples.

This study addresses a critical limitation of traditional multiple testing procedures—such as the Benjamini–Hochberg (BH) method—which control the overall false discovery rate (FDR) but offer no guarantee regarding the reliability of boundary discoveries, i.e., the least significant rejections. The authors propose a novel two-stage adaptive approach: first estimating the number of true null hypotheses using non-significant test statistics, then applying an adjusted threshold within the Support Line (SL) framework to control the error probability of boundary discoveries. This work is the first to integrate adaptivity into boundary FDR control, providing rigorous error guarantees under independence and demonstrating robustness and enhanced power under positive dependence. Theoretical analysis confirms its validity, simulations show substantially improved statistical power over the original SL procedure, and real-world applicability is illustrated through a meta-analysis in psychology.

This work aims to enhance the statistical power of adaptive Benjamini–Hochberg (BH) procedures while maintaining control of the false discovery rate (FDR). By unifying existing adaptive FDR methods under a common framework—interpreting them as weighted BH procedures based on composite e-values (ep-BH)—the study reveals their shared structural foundation and demonstrates for the first time that most estimators of the proportion of true null hypotheses inherently correspond to composite e-values. Building on this insight, the authors propose a novel framework that uniformly improves upon nearly all existing methods without requiring additional assumptions, and they develop a new ep-BH procedure with finite-sample FDR guarantees. In canonical settings such as t-tests, the proposed method achieves consistent and robust power gains while rigorously controlling the FDR.

This study addresses the challenge that conventional variable selection methods struggle to control the false discovery rate (FDR) when predictors are highly correlated and often erroneously discard entire groups of related variables, thereby impairing predictive performance. To overcome this, the authors propose a hierarchical ensemble variable selection framework: variables are first grouped via hierarchical clustering, and each group is then tested for the presence of any non-zero effect, allowing any member to serve as a proxy. This approach uniquely integrates ensemble selection with hierarchical clustering into an FDR-controlling framework—extending beyond prior methods limited to family-wise error rate (FWER) control—and employs a generalized Benjamini–Hochberg/Yekutieli step-up procedure to account for logical dependencies among composite hypotheses. Simulations and empirical analyses demonstrate that the method achieves strict FDR control while substantially improving statistical power, yielding richer and more predictive variable selections.

This study addresses the challenge in multiple hypothesis testing where existing methods struggle with unknown and arbitrary dependence structures among p-values, thereby limiting predictive power analysis and sample size planning. The authors propose the first Bayesian predictive power framework that accommodates arbitrary dependence without requiring independence assumptions, while supporting control of either the family-wise error rate (FWER) or the false discovery rate (FDR). By incorporating prior distributions on effect sizes, a uniform prior on the correlation matrix, and p-value weighting, the approach effectively mitigates p-hacking bias. Inference is carried out via Bayesian simulation using an asymmetric multivariate normal mean-variance mixture distribution with a scale-matrix mixture and a Dirichlet process prior, implemented in the R package bnpMTP. Application to a reanalysis of p-values from a lead exposure study demonstrates more robust power estimation and bias assessment, offering a reliable foundation for future sample size determination.

This work addresses a limitation of conventional approaches in multiple hypothesis testing within location families, which typically control the false discovery rate (FDR) only under the global null hypothesis. In practice, however, there is often a need to control FDR uniformly over non-significant regions across the entire parameter space. The paper reframes FDR as a function of the location parameter and proposes a natural extension of the Benjamini–Hochberg (BH) procedure that simultaneously controls the entire FDR curve without incurring additional computational cost. Theoretical analysis establishes that the proposed method guarantees the FDR curve remains below a user-specified level uniformly, and numerical experiments corroborate its effectiveness and practical utility.