Stochastic Grouping Conformal Prediction for Effective Subgroup Reliability

2026-10-08

AI Summary

This study addresses the challenges of uneven subgroup coverage, over-reliance on sensitive attributes, and prediction set inflation caused by worst-group bottlenecks in conformal prediction. To this end, we propose a randomized grouped conformal prediction framework that introduces learned stochastic grouping mappings to replace fixed predefined groups. This enables each sample to aggregate information from neighbors with similar calibration behavior and generate local scoring rules, thereby enhancing subgroup reliability while preserving standard marginal coverage guarantees. Experiments on both synthetic and real-world benchmarks demonstrate that the proposed framework significantly reduces subgroup coverage disparities and effectively mitigates dependence on sensitive attributes. Furthermore, it yields prediction set sizes that are comparable to or smaller than those of existing baseline methods.

Abstract

Conformal prediction offers a distribution-free coverage guarantee, making it especially attractive for clinical applications. Standard conformal prediction, however, provides such guarantees only at the population level, and its prediction sets can exhibit coverage disparities across clinically important subgroups. A natural remedy is to calibrate within predefined groups. However, this can require access to sensitive subgroup attributes and is prone to a worst-group bottleneck: protecting the most difficult subgroup can inflate prediction sets for all, increasing cognitive burden on decision makers. To this end, we propose Stochastic Grouping Conformal Prediction (SGCP), a conformal framework for subgroup-reliable uncertainty quantification. It learns a stochastic grouping map that allows each sample to draw calibration information from others with similar calibration behavior, yielding a local score law that boosts reliability across subpopulations. We prove that SGCP retains the standard coverage guarantee. Experiments on synthetic and real-world benchmarks show that it consistently reduces subgroup coverage gaps while achieving smaller or comparable prediction set sizes relative to existing baselines.

Problem

• Conformal Prediction
• Subgroup Reliability
• Coverage Disparity
• Uncertainty Quantification
• Worst-group Bottleneck

Innovation

• Conformal Prediction
• Stochastic Grouping
• Uncertainty Quantification
• Subgroup Reliability
• Coverage Guarantee