When Is a Conformal Guarantee Fair? Auditing Silent Subgroup Under-Coverage in Alzheimer's Disease Longitudinal Prediction

2026-08-04

AI Summary

Standard conformal prediction in longitudinal Alzheimer’s disease forecasting guarantees only marginal coverage, often overlooking undercoverage in high-risk subgroups. This work systematically disentangles two underlying mechanisms—subgroup rarity and heavy-tailed outcome distributions—and proposes tailored correction strategies that enforce a coverage safety margin to ensure reliability. By integrating cross-conformal pooling with subgroup-specific calibration, the method undergoes multidimensional auditing on ADNI and OASIS-3 datasets. Empirical evaluation reveals coverage deficits in 57 out of 68 high-risk subgroup settings; after correction, coverage consistently recovers to the target level, substantially enhancing prediction credibility for genetically susceptible individuals and patients with severe disease progression.

Abstract

Longitudinal prediction of Alzheimer's disease biomarkers increasingly informs clinical decisions, and a forecast is only useful if it also reports how much to trust it. Conformal prediction supplies this by wrapping any forecaster in a prediction band with a finite-sample coverage guarantee under exchangeability. However, standard population-level conformal prediction guarantees only marginal coverage and may mask substantial under-coverage within clinically important subgroups. We introduce a general mechanism-driven framework for auditing and repairing such subgroup under-coverage. Across two cohorts (ADNI, OASIS-3), two base forecasters, and nine attributes spanning genetic risk, demographics, and clinical severity, we find that population-level bands under-cover high-risk subgroups in 57 of 68 audited combinations, despite achieving nominal marginal coverage. We trace these failures to two mechanisms: (A) \emph{rarity}, where a group-conditional band calibrated on only $n$ patients covers at most $k/(n+1)$; and (B) \emph{tail-heaviness}, where a population-wide band is too narrow for a heavy-tailed subgroup and additional data cannot close the gap. Under-coverage falls disproportionately on patients with high genetic risk and disease severity (6.1 pp mean deficit, 95\% CI [3.3, 8.9]), while demographic groups remain at the target level on average (0.0 pp, CI [$-1.9$, 1.7]). We pair each mechanism with a corresponding conformal correction: cross-conformal pooling for rarity, per-subgroup calibration for tail-heaviness, and a coverage-safe marginal floor when both arise. Together, these corrections restore target coverage for nearly every high-risk subgroup across both cohorts and forecasters.

Problem

• conformal prediction
• subgroup under-coverage
• Alzheimer's disease
• longitudinal prediction
• fairness

Innovation

• conformal prediction
• subgroup under-coverage
• mechanism-driven auditing
• cross-conformal pooling
• per-subgroup calibration