Randomness, exchangeability, and conformal prediction

2025-01-20

AI Summary

This paper addresses the lack of a unified theoretical framework for uncertainty quantification under non-IID data. Methodologically, it introduces two most general classes of confidence predictors—stochasticity-based predictors and exchangeability-based predictors—grounded respectively in functional stochasticity theory and the exchangeability assumption. A universal confidence prediction framework is developed, applicable to arbitrary sequences (including both IID and exchangeable sequences), with a rigorous derivation of its deviation upper bound relative to conformal prediction. Theoretical contributions are twofold: first, it establishes, for the first time, quantitative approximation guarantees—under controllable error—between both predictor classes and conformal prediction at *any* finite sample size, thereby providing a rigorous foundation for uncertainty estimation on non-independent data; second, it unifies the intrinsic relationships among stochasticity, exchangeability, and conformality, significantly broadening the applicability and deepening the theoretical foundations of uncertainty estimation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

This note continues development of the functional theory of randomness, a modification of the algorithmic theory of randomness getting rid of unspecified additive constants. It introduces new kinds of confidence predictors, including randomness predictors (the most general confidence predictors based on the assumption of IID observations) and exchangeability predictors (the most general confidence predictors based on the assumption of exchangeable observations). The main result implies that both are close to conformal predictors and quantifies the difference between them.

Problem

• Stochastic Theory
• Predictive Methods
• Quantification of Differences

Innovation

• Stochastic Predictability
• Exchangeable Predictability
• Consistent Prediction Comparison