Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models

2025-08-11

AI Summary

Large language models (LLMs) frequently generate fluent yet factually incorrect “hallucinations,” with reliability degradation exacerbated in multi-turn interactions. A critical gap lies in understanding whether LLMs can self-identify erroneous outputs and how contextual factors induce misalignment between confidence and correctness. Method: We propose a token-level probing framework that quantifies aleatoric (data-driven) and epistemic (model-knowledge) uncertainties directly from logits. Unreliable tokens are localized via uncertainty thresholds, and their hidden states are aggregated to derive response-level reliability scores—requiring no external annotations or fine-tuning. Contribution/Results: Evaluated on open-domain question answering benchmarks, our method significantly improves hallucination detection over baseline uncertainty metrics. It exposes fundamental limitations of raw uncertainty signals and establishes an interpretable, model-agnostic internal reliability assessment mechanism—enabling seamless integration into trustworthy generation pipelines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) are prone to generating fluent but incorrect content, known as confabulation, which poses increasing risks in multi-turn or agentic applications where outputs may be reused as context. In this work, we investigate how in-context information influences model behavior and whether LLMs can identify their unreliable responses. We propose a reliability estimation that leverages token-level uncertainty to guide the aggregation of internal model representations. Specifically, we compute aleatoric and epistemic uncertainty from output logits to identify salient tokens and aggregate their hidden states into compact representations for response-level reliability prediction. Through controlled experiments on open QA benchmarks, we find that correct in-context information improves both answer accuracy and model confidence, while misleading context often induces confidently incorrect responses, revealing a misalignment between uncertainty and correctness. Our probing-based method captures these shifts in model behavior and improves the detection of unreliable outputs across multiple open-source LLMs. These results underscore the limitations of direct uncertainty signals and highlight the potential of uncertainty-guided probing for reliability-aware generation.

Problem

• Detecting unreliable LLM outputs to reduce confabulation risks
• Assessing impact of in-context information on model reliability
• Improving uncertainty-guided probing for reliability-aware generation

Innovation

• Token-level uncertainty guides representation aggregation
• Aleatoric and epistemic uncertainty from logits
• Probing-based method detects unreliable outputs