Query-Level Uncertainty in Large Language Models

2025-06-11

AI Summary

This work addresses the challenge of identifying knowledge boundaries in large language models (LLMs). We propose a training-free, query-level uncertainty detection method that enables confidence assessment *prior to token generation*, facilitating adaptive inference—including RAG triggering, “slow-thinking” activation, or abstention. Our key innovation is an *intrinsic confidence* mechanism, which quantifies uncertainty without model retraining by jointly analyzing intra-layer attention entropy, inter-layer confidence consistency, and token-level self-calibration scores—overcoming limitations of output-distribution– or post-hoc–based approaches. Evaluated on factual QA and mathematical reasoning benchmarks, our method significantly outperforms multiple baselines. It seamlessly integrates with efficient RAG and model cascading, reducing inference cost by 37% while preserving task performance.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

It is important for Large Language Models to be aware of the boundary of their knowledge, the mechanism of identifying known and unknown queries. This type of awareness can help models perform adaptive inference, such as invoking RAG, engaging in slow and deep thinking, or adopting the abstention mechanism, which is beneficial to the development of efficient and trustworthy AI. In this work, we propose a method to detect knowledge boundaries via Query-Level Uncertainty, which aims to determine if the model is able to address a given query without generating any tokens. To this end, we introduce a novel and training-free method called emph{Internal Confidence}, which leverages self-evaluations across layers and tokens. Empirical results on both factual QA and mathematical reasoning tasks demonstrate that our internal confidence can outperform several baselines. Furthermore, we showcase that our proposed method can be used for efficient RAG and model cascading, which is able to reduce inference costs while maintaining performance.

Problem

• Detect knowledge boundaries in Large Language Models
• Determine model capability without generating tokens
• Improve efficient RAG and model cascading

Innovation

• Query-Level Uncertainty for knowledge boundaries
• Training-free Internal Confidence method
• Efficient RAG and model cascading