Conformalized Large Language Models under Configuration Shift

2026-08-02

AI Summary

This work addresses a previously overlooked issue—“configuration shift”—wherein the coverage validity of conformal predictions in large language models (LLMs) degrades significantly under common configuration changes such as prompt templates, decoding temperatures, or weight quantization. The study formalizes this phenomenon, showing that configuration shifts disrupt the consistency between the nonconformity score distributions used during calibration and testing, thereby undermining coverage guarantees. To mitigate this, the authors derive a theoretical lower bound on coverage and propose two practical solutions: a vulnerability-aware calibration ensemble that requires no test data and a boundary-based recalibration method. Extensive experiments across nine LLMs, four datasets, and four scoring mechanisms demonstrate that the proposed approaches effectively restore target coverage, particularly when test samples are scarce or unavailable, without compromising prediction efficiency.

Abstract

Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample coverage guarantees under exchangeability. Yet for LLMs, nonconformity scores are often induced by an inference pipeline, not just a fixed model, making them depend not only on the data distribution but also on configurable factors such as the prompt template, decoding parameters, and deployment setting. Since such configurations are routinely modified in practice but rarely treated as a source of shift, their impact on CP validity remains poorly understood. We call this \emph{configuration shift} and study it systematically along three axes: prompt template, decoding temperature, and weight quantization. In a broad empirical study spanning $9$ LLMs, $4$ datasets, and $4$ nonconformity scores, we find that configuration shift consistently erodes CP validity, often driving empirical coverage below the target. By contrast, efficiency is largely preserved: valid prediction sets remain close in size to the i.i.d. baseline. We derive coverage lower bounds that attribute this loss to a discrepancy between calibration and test score distributions, and use their finite-sample plug-in versions as empirical diagnostics of shift severity. We further show that these findings lead to practical mitigations: bound-inspired recalibration is effective with limited test examples, while fragility-aware calibration ensembling recovers much of the lost coverage without test data.

Problem

• conformal prediction
• configuration shift
• large language models
• uncertainty quantification
• coverage validity

Innovation

• configuration shift
• conformal prediction
• large language models
• uncertainty quantification
• coverage validity