🤖 AI Summary
Existing conformal prediction (CP) methods provide finite-sample validity guarantees but lack the ability to quantify support strength for arbitrary events and do not support posterior inference.
Method: We propose model-free generalized fiducial inference (MF-GFI), the first framework unifying desiderata-based inference with confidence set approximation via optimal probability measures that approximate fuzzy belief/likelihood pairs, with reliability and accountability as core inferential principles. The method strictly controls Type-I error in finite samples and enables both exact and approximate probabilistic reasoning. We develop a computationally tractable probability approximation algorithm yielding prediction sets with rigorous error guarantees.
Contribution/Results: This work establishes the first model-free, accountable, and broadly applicable statistical framework for uncertainty quantification in machine learning—offering finite-sample validity, event-specific support assessment, and principled posterior-like inference without distributional assumptions.
📝 Abstract
Motivated by the need for the development of safe and reliable methods for uncertainty quantification in machine learning, I propose and develop ideas for a model-free statistical framework for imprecise probabilistic prediction inference. This framework facilitates uncertainty quantification in the form of prediction sets that offer finite sample control of type 1 errors, a property shared with conformal prediction sets, but this new approach also offers more versatile tools for imprecise probabilistic reasoning. Furthermore, I propose and consider the theoretical and empirical properties of a precise probabilistic approximation to the model-free imprecise framework. Approximating a belief/plausibility measure pair by an [optimal in some sense] probability measure in the credal set is a critical resolution needed for the broader adoption of imprecise probabilistic approaches to inference in statistical and machine learning communities. It is largely undetermined in the statistical and machine learning literatures, more generally, how to properly quantify uncertainty in that there is no generally accepted standard of accountability of stated uncertainties. The research I present in this manuscript is aimed at motivating a framework for statistical inference with reliability and accountability as the guiding principles.