Conformal Language Modeling via Posterior Sampling

2026-06-02

AI Summary

This work addresses the challenge of hallucinations in large language model (LLM) generation, which existing post-hoc correction methods often mitigate at the cost of reduced coherence or deviation from the original output distribution. The paper introduces a novel approach that integrates conformal prediction directly into the generative process rather than applying it as a post-processing step. By leveraging a calibration mechanism tailored for conditional sequence generation and employing approximate posterior sampling, the method performs generation within high-confidence regions while preserving statistical validity. This strategy maintains rigorous risk control guarantees while significantly enhancing the coherence and practical utility of generated text. Experimental results on biographical generation and mathematical problem-solving tasks demonstrate that the proposed method achieves substantially higher output quality than existing techniques, all while providing comparable statistical assurances.

Abstract

Large Language Models remain plagued by hallucinations. Recent work has sought to tame their prevalence using statistical techniques based on conformal prediction, with both theoretical and empirical success. However, these methods operate in a post-hoc fashion, treating the sampling procedure itself as atomic and then surgically altering samples to remove hallucinated claims. This disconnect between filtering and generation can result in samples that are incoherent, inconsistent, or simply unlikely under the model itself. Moreover, post-hoc surgery is unable to shift probability mass towards more useful and helpful responses. To address these issues, we propose to instead sample from approximations to an LLM posterior, where the conditioning event corresponds to a calibrated, high-scoring region. We develop a calibration procedure tailored to the setting of conditional sequential generation that effectively identifies this region and achieves target risk control. Empirically, we apply our method to case studies focused on open-ended biography generation and mathematical problem solving; compared to prior work, we obtain the same statistical guarantees, with higher downstream utility.

Problem

• hallucinations
• conformal prediction
• post-hoc correction
• language generation
• coherence

Innovation

• conformal prediction
• posterior sampling
• language modeling
• hallucination mitigation
• calibrated generation