Conformal Prediction under Exponential-Tilt Joint Shift

2026-09-25

AI Summary

This study addresses the failure of conformal prediction coverage guarantees caused by post-deployment data distribution shifts. It proposes Exponential Tilting Reweighting Alignment (ExTRA), a framework for modeling distribution shifts that systematically compares two adaptation strategies: weight calibration and prediction tilting. The analysis reveals that while prediction tilting can reduce set sizes under specific conditions, it may compromise coverage validity. Experiments demonstrate that ExTRA decreases prediction set lengths by 30% in synthetic regression tasks; however, it induces significant coverage degradation in classification or information-deficient settings and yields no consistent benefits on real-world data. Determining the precise applicability conditions for this approach remains an open challenge.

Abstract

Conformal prediction can lose coverage when the data distribution changes after deployment. We study adaptation using labeled source data and unlabeled target inputs, allowing both the input distribution and its relationship with outcomes to change. We use Exponential Tilt Reweighting Alignment (ExTRA), introduced for classification by Maity et al. (2023), to estimate structured distribution shifts. We compare using its estimated weights in conformal calibration with additionally tilting the source predictive distribution. Shared learned predictors, estimated weights, calibration samples, and test observations isolate the effect of tilting. Existing theory gives both procedures target coverage with true weights and a common coverage bound with estimated weights. Identification calculations and an analysis of how scoring interacts with weight estimation error help explain why their performance can nevertheless differ. In a synthetic regression setting where the assumed models match the data-generating process and target inputs are informative about the shift, tilting reduces mean set length by about $30\%$ relative to weighting alone, with both methods attaining coverage near nominal. Tilting can instead cause substantial coverage losses in synthetic classification and in regression when target inputs provide little information about the response shift. Real-data experiments also show no consistent benefit. Good coverage from weighted calibration alone does not ensure that adding predictive tilting will preserve coverage. Deciding when to apply this additional adjustment using only source labels and target inputs remains an open problem.

Problem

• Conformal Prediction
• Distribution Shift
• Coverage Guarantee
• Exponential Tilt
• Domain Adaptation

Innovation

• Conformal Prediction
• Distribution Shift
• Exponential Tilt Reweighting
• Domain Adaptation
• Predictive Tilting