apply conformal prediction

Design, build, and evaluate procedures that convert model outputs and nonconformity scores into distribution‑free prediction sets, intervals, or bands with finite‑sample marginal or conditional coverage guarantees; this includes adapting calibration algorithms for group‑conditional guarantees, online/streaming settings, weighted or label‑weighted schemes, selection‑corrected and shift‑aware adjustments, and time‑series, spatial‑temporal or spectral variants. Analyze and select calibration strategies and nonconformity scores, quantify empirical coverage and concentration of failures, and prove or test guarantees under selection, dataset shift, and impossibility bounds.

applyconformalprediction

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
0.37
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$200K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

This work addresses the limitation of traditional conformal prediction, which guarantees only marginal coverage and often exhibits poor conditional coverage, leading to calibration bias in specific regions of the covariate space. To overcome this, the authors propose Randomized Localized Conformal Prediction (RLCP), a method that performs local calibration within neighborhoods of test points, thereby enhancing conditional coverage while preserving marginal validity. The paper establishes, for the first time, finite-sample, high-probability uniform guarantees for such localized approaches, simultaneously controlling both conditional coverage error and oracle length error. By leveraging Hölder continuity, kernel density estimation, data-splitting-based score learning, and conformal quantile regression, the authors develop a theoretical framework for local coverage, deriving finite-sample bounds on the conditional coverage gap and length error, and demonstrating that improved score estimation enables performance approaching that of the oracle.

conditional coverageconformal predictionfinite-sample guarantees

This work addresses the failure of traditional conformal prediction to maintain coverage guarantees under distribution shift. Focusing on bounded label-conditional covariate shift, the authors propose a source-tuned pseudo-calibration algorithm that interpolates between hard pseudo-labels and random labels, guided by classifier uncertainty. A relaxation parameter dynamically adjusts the conformity threshold to preserve a pre-specified target-domain coverage rate. By integrating domain adaptation, Wasserstein distance metrics, and conformal prediction, the method establishes a theoretical lower bound on target-domain coverage and provides a qualitative characterization of pseudo-calibration behavior. Experimental results demonstrate that the proposed approach effectively mitigates coverage degradation caused by distribution shift while maintaining reasonably sized prediction sets.

conformal predictioncoverage guaranteedistribution shift

This paper addresses the conditional coverage performance of prediction intervals in regression. We propose a “conjecture testing” framework that imports hypothesis-testing principles into predictive inference; introduce— for the first time in nonparametric regression—the notion of *pertinence*, quantifying how well a prediction interval aligns with the local data distribution; and establish theoretically that model-free bootstrap achieves superior conditional coverage compared to quantile regression and substantially outperforms standard conformal prediction under mild regularity conditions. To enhance finite-sample reliability, we incorporate uniformization transformations and a refined conformal scoring function. Empirical results demonstrate significant improvements in interval pertinence under limited samples and support one-sided conjecture testing—thereby addressing key limitations of conformal prediction in both conditional coverage calibration and directional inference.

Compare Model-free Bootstrap and conformal prediction performanceEvaluate conditional coverage of prediction intervalsExtend pertinence concept to nonparametric regression

Distribution-free inference with hierarchical data

Jun 10, 2023
YL
Yonghoon Lee
🏛️ University of Pennsylvania | University of Chicago

This work addresses the failure of standard exchangeability—and consequent unreliability of distribution-free inference—under hierarchical data structures (e.g., grouped or repeated-measures designs). We introduce *hierarchical exchangeability*, the first formal theoretical foundation for distribution-free inference in non-i.i.d. hierarchical settings. Methodologically, we extend conformal prediction and the jackknife+ framework to hierarchical structures and propose a second-moment coverage mechanism, strengthening guarantees from marginal coverage to *conditional second-moment coverage*. Experiments demonstrate that our approach substantially reduces conditional miscoverage rates; under model misspecification, prediction interval width increases only marginally, while under correct model specification, the overhead is negligible. Our core contribution is the first provably reliable, distribution-free inference framework tailored specifically for hierarchical data.

Achieves stronger second-moment coverage for repeated measurementsBalances prediction set width and conditional miscoverage ratesExtends distribution-free methods for hierarchical data structures

Distribution-Free Calibration of Statistical Confidence Sets

Nov 28, 2024
LM
Luben Miguel Cruz Cabezas
🏛️ Federal University of S~ao Carlos | University of S~ao Paulo

In statistical inference, confidence sets—especially under complex models or small sample sizes—often fail to achieve nominal coverage levels, particularly in likelihood-free inference (LFI) settings. To address this, we propose TRUST and TRUST++, two distribution-free, simulation-based calibration methods that adapt conformal prediction principles to confidence set construction with redundant parameters, thereby establishing the first distribution-agnostic calibration framework for statistical inference. Our methods guarantee finite-sample local coverage and asymptotic conditional coverage, while enabling self-assessment of simulation cost. Theoretically, we prove their robustness against model misspecification and simulation imperfection. Empirically, TRUST and TRUST++ significantly improve coverage accuracy across both tractable and intractable likelihood models, consistently outperforming existing approaches—especially in small-sample regimes.

Achieving distribution-free conditional coverage with simulationsCalibrating confidence sets for valid statistical inferenceHandling nuisance parameters in small-sample complex models

Latest Papers

What's happening recently
View more

This work addresses the limitations of traditional conformal prediction, which relies on data exchangeability and struggles with non-exchangeable time series exhibiting seasonality, periodicity, or time-varying structures. The authors propose a spectral adaptive conformal prediction method that constructs weighted quantiles based on local spectral similarity and incorporates an online miscoverage rate calibration mechanism. This approach preserves finite-sample coverage guarantees while effectively capturing dynamic uncertainty in structured non-exchangeable data. By integrating spectral analysis, weighted conformal prediction, and effective sample size diagnostics, the method demonstrates superior performance over fixed spectral weighting strategies across simulations and three real-world U.S. datasets, confirming its robustness and reliability in handling complex temporal dependencies.

conformal predictionnon-exchangeable dataprediction intervals

Existing online conformal prediction methods struggle to simultaneously achieve parameter-free operation and rigorous error control under group-conditional settings in non-stationary data streams, limiting the fairness and robustness of uncertainty quantification. This work proposes the first online conformal prediction algorithm that unifies parameter-free online optimization with group-conditional coverage guarantees. The method requires no learning rate tuning and provides strict conditional coverage for distinct data groups even in dynamic environments. Empirical evaluations demonstrate that the proposed approach substantially improves the reliability of current parameter-free methods while delivering prediction interval quality comparable to carefully tuned group-conditional baselines.

distribution shiftgroup-conditional coverageonline conformal prediction

Traditional bootstrap and conformal prediction methods fail in time series settings due to violations of exchangeability and the absence of a unified framework that supports dependence-aware resampling and adaptive conformal calibration. This work proposes the first typed API integrating block, residual, sieve, and wild resampling schemes with adaptive conformal approaches such as EnbPI and ACI, enabling distribution-free uncertainty quantification. Leveraging compilation-based acceleration and streaming reductions, the method requires only O(B) additional memory, circumventing the O(Bn) tensor duplication typical of conventional implementations. Empirical results demonstrate that the approach substantially mitigates undercoverage under the i.i.d. assumption, with sieve resampling achieving coverage closest to the nominal level for short-memory linear processes, while running several times faster than the arch benchmark.

bootstrapconformal predictiondependence

This work addresses the challenge of conformal prediction under distributional shifts and structural changes in non-stationary streaming data, where traditional methods relying on exchangeability assumptions often fail. To overcome this limitation, the authors propose the DASC framework, which uniquely integrates local spectral similarity with an optimal transport–based drift score to dynamically weight residuals and adaptively adjust both the calibration pool and the target miscoverage level. This approach enables adaptive uncertainty quantification for non-exchangeable streaming data and introduces an online effective sample size diagnostic to assess predictive robustness. Experimental results on synthetic and real-world datasets—including electricity load, weather, and financial time series—demonstrate that DASC consistently achieves nominal or conservative coverage while reducing average prediction interval width by 28%–42% compared to existing methods.

conformal predictiondistributional driftnon-exchangeable data

Hot Scholars

OS

Osvaldo Simeone

King's College London
Information theorymachine learningquantum information processingwireless systems
LL

Lars Lindemann

Assistant Professor of Algorithmic Systems Theory, ETH Zürich
Systems and Control TheoryFormal MethodsMachine LearningAutonomous Systems
AR

Aaditya Ramdas

Associate Professor (with tenure), Carnegie Mellon University
Machine LearningStatistics
MZ

Matteo Zecchin

King's College London
Wireless CommunicationMachine LearningDistributed OptimizationBayesian Learning
ZZ

Zhixin Zhou

Alpha Benito Research
StatisticsMachine Learning