🤖 AI Summary
This work addresses the problem of exact calibration of conditional confidence sets in simulation-based inference—specifically, accurately modeling the conditional sampling distribution (or p-value function) of a scalar test statistic given compressed observations, to guarantee strict conditional coverage. To this end, we propose an automatic-differentiation-driven neural network method that directly learns the one-dimensional conditional cumulative distribution function (CDF) of the test statistic; its derivative, computed via backpropagation, serves as a differentiable approximation of the conditional probability density function (PDF), unifying CDF modeling and PDF estimation for the first time. Leveraging uncertainty quantification techniques—including Monte Carlo Dropout, ensemble averaging, and quantile regression—we substantially improve coverage reliability. Experiments demonstrate that our approach strictly achieves the target conditional coverage across multiple tasks, while significantly reducing modeling complexity and sampling overhead compared to state-of-the-art density-ratio estimation baselines.
📝 Abstract
Simulation-based inference methods that feature correct conditional coverage of confidence sets based on observations that have been compressed to a scalar test statistic require accurate modeling of either the p-value function or the cumulative distribution function (cdf) of the test statistic. If the model of the cdf, which is typically a deep neural network, is a function of the test statistic then the derivative of the neural network with respect to the test statistic furnishes an approximation of the sampling distribution of the test statistic. We explore whether this approach to modeling conditional 1-dimensional sampling distributions is a viable alternative to the probability density-ratio method, also known as the likelihood-ratio trick. Relatively simple, yet effective, neural network models are used whose predictive uncertainty is quantified through a variety of methods.