🤖 AI Summary
This study addresses the multiple testing problem in matched observational studies with a single intervention and multiple endpoints. We propose a robust method that jointly controls the false discovery rate (FDR) and quantifies unmeasured confounding bias. Our key innovation is the first integration of FDR control with formal sensitivity analysis, achieved via integer programming and a hierarchical screening strategy to efficiently compute sensitivity sets—i.e., subsets of hypotheses remaining significant under varying magnitudes of unmeasured confounding—enabling conservative estimation of the true positive rate (TPR). The method supports simultaneous inference across the entire hypothesis space, balancing statistical power and robustness. Simulation studies and an empirical application investigating long-term effects of childhood abuse demonstrate that our approach reliably identifies high-confidence endpoint subsets even under substantial hidden bias, substantially improving the reproducibility and interpretability of exploratory analyses.
📝 Abstract
We provide an approach to exploratory data analysis in matched observational studies with a single intervention and multiple endpoints. In such settings, the researcher would like to explore evidence for actual treatment effects among these variables while accounting not only for the possibility of false discoveries, but also for the potential impact of unmeasured confounding. For any candidate subset of hypotheses about these outcomes, we provide sensitivity sets for the proportion of the hypotheses within the subset which are actually true. The resulting sensitivity statements are valid simultaneously over all possible choices for the rejected set, allowing the researcher to search for promising subsets of hypotheses that maintain a large estimated fraction of true discoveries even if hidden bias is present. The approach is well suited to sensitivity analysis, as conclusions that some fraction of outcomes are affected by the treatment exhibit larger robustness to unmeasured confounding than findings that any particular outcome is affected. We show how a sequence of integer programs, in tandem with screening steps, facilitate the efficient computation of the required sensitivity sets. We illustrate the practical utility of our method through both simulation studies and a data example on the long-term impacts of childhood abuse.