Robust Hallucination Detection in LLMs via Adaptive Token Selection

2025-04-10

AI Summary

Existing hallucination detection methods for large language models (LLMs) suffer from poor robustness—particularly under variable-length and sparse hallucination distributions—due to their reliance on fixed internal representations, leading to significant performance fluctuations. To address this, we propose HaMI, a robust hallucination detection framework based on adaptive key-token selection. HaMI formulates hallucination detection as a sequence-level multiple-instance learning (MIL) task, jointly optimizing token-level representation selection and hallucination classification in an end-to-end differentiable manner. Crucially, HaMI introduces the first dynamic token selection mechanism, eliminating dependence on predefined positions or static token sets and enabling deeper exploitation of LLMs’ internal semantic representations. Evaluated on four mainstream hallucination benchmarks, HaMI substantially outperforms state-of-the-art methods, achieving significant gains in both detection accuracy and cross-domain generalization capability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Hallucinations in large language models (LLMs) pose significant safety concerns that impede their broader deployment. Recent research in hallucination detection has demonstrated that LLMs' internal representations contain truthfulness hints, which can be harnessed for detector training. However, the performance of these detectors is heavily dependent on the internal representations of predetermined tokens, fluctuating considerably when working on free-form generations with varying lengths and sparse distributions of hallucinated entities. To address this, we propose HaMI, a novel approach that enables robust detection of hallucinations through adaptive selection and learning of critical tokens that are most indicative of hallucinations. We achieve this robustness by an innovative formulation of the Hallucination detection task as Multiple Instance (HaMI) learning over token-level representations within a sequence, thereby facilitating a joint optimisation of token selection and hallucination detection on generation sequences of diverse forms. Comprehensive experimental results on four hallucination benchmarks show that HaMI significantly outperforms existing state-of-the-art approaches.

Problem

• Detect hallucinations in large language models robustly
• Adaptively select critical tokens for hallucination detection
• Improve detection performance on diverse generation sequences

Innovation

• Adaptive token selection for hallucination detection
• Multiple Instance learning over token-level representations
• Joint optimization of token selection and detection