🤖 AI Summary
This paper addresses the conditional coverage performance of prediction intervals in regression. We propose a “conjecture testing” framework that imports hypothesis-testing principles into predictive inference; introduce— for the first time in nonparametric regression—the notion of *pertinence*, quantifying how well a prediction interval aligns with the local data distribution; and establish theoretically that model-free bootstrap achieves superior conditional coverage compared to quantile regression and substantially outperforms standard conformal prediction under mild regularity conditions. To enhance finite-sample reliability, we incorporate uniformization transformations and a refined conformal scoring function. Empirical results demonstrate significant improvements in interval pertinence under limited samples and support one-sided conjecture testing—thereby addressing key limitations of conformal prediction in both conditional coverage calibration and directional inference.
📝 Abstract
Predictive inference under a general regression setting is gaining more interest in the big-data era. In terms of going beyond point prediction to develop prediction intervals, two main threads of development are conformal prediction and Model-free prediction. Recently, a new conformal prediction approach was proposed that exploits the same uniformization procedure as in the well-known Model-free Bootstrap. Hence, it is of interest to compare and further investigate the performance of the two methods. In the paper at hand, we contrast the two approaches via theoretical analysis and numerical experiments with a focus on conditional coverage of prediction intervals. We discuss suitable scenarios for applying each algorithm, underscore the importance of conditional vs. unconditional coverage, and show that, under mild conditions, the Model-free bootstrap yields prediction intervals with guaranteed better conditional coverage compared to quantile estimation. We also extend the concept of 'pertinence' of prediction intervals to the nonparametric regression setting, and give concrete examples where its importance emerges under finite sample scenarios. Finally, we define the new notion of 'conjecture testing' that is the analog of hypothesis testing as applied to the prediction problem; we also devise a modified conformal score to allow conformal prediction to handle one-sided 'conjecture tests', and compare to the Model-free bootstrap.