🤖 AI Summary
This study addresses a critical limitation of traditional multiple testing procedures—such as the Benjamini–Hochberg (BH) method—which control the overall false discovery rate (FDR) but offer no guarantee regarding the reliability of boundary discoveries, i.e., the least significant rejections. The authors propose a novel two-stage adaptive approach: first estimating the number of true null hypotheses using non-significant test statistics, then applying an adjusted threshold within the Support Line (SL) framework to control the error probability of boundary discoveries. This work is the first to integrate adaptivity into boundary FDR control, providing rigorous error guarantees under independence and demonstrating robustness and enhanced power under positive dependence. Theoretical analysis confirms its validity, simulations show substantially improved statistical power over the original SL procedure, and real-world applicability is illustrated through a meta-analysis in psychology.
📝 Abstract
A cornerstone of the multiple testing literature is the Benjamini-Hochberg (BH) procedure, which guarantees control of the FDR when $p$-values are independent or positively dependent. While BH controls the average quality of rejections, it does not provide guarantees for individual discoveries, particularly those near the rejection threshold, which are more likely to be false than the average rejection. For independent $p$-values with Uniform$(0,1)$ null distribution, the Support Line procedure (SL; arXiv:2207.07299) provably controls the error probability for the rejection at the edge of the discovery set (i.e. the one with largest $p$-value) at level $q m_0/m$, where $m_0$ is the number of true null hypotheses and $q$ is a tuning parameter. In this work, we study adaptive versions of the SL procedure that operate in two steps: the first step estimates $m_0$ from non-significant statistics, and the second step runs the SL procedure at an adjusted level $q m / \hat{m}_0$. The adaptive procedures are shown to control the false discovery probability for the "boundary'' rejection under an independence assumption. Simulation studies suggest that some but not all of the two-stage procedures maintain error control under positive dependence, and that substantial power is gained relative to the original SL procedure. We illustrate differences between the procedures on meta-data from the recent literature in behavioral psychology on growth mindset and nudge interventions.