Diversity-Oriented Fine-Tuning for Uncertainty-Based Hallucination Detection

2026-07-18

AI Summary

This work addresses the limitation of existing hallucination detection methods, which predominantly rely on post-hoc processing during inference and struggle to identify repetitive errors characterized by low semantic entropy. To overcome this, the study introduces generation diversity into the training phase for the first time, proposing two fine-tuning strategies based on supervised fine-tuning (SFT) and direct preference optimization (DPO). These approaches jointly enhance model generation diversity and its semantic entropy response to hallucinatory content. By co-optimizing diversity and detectability, the method significantly reduces low-entropy hallucinated outputs and achieves state-of-the-art or competitive performance across multiple benchmarks in hallucination detection.

Abstract

Existing hallucination detection methods are typically conducted at the inference stage, without making any modifications to the model itself. In this paper, we are interested in exploring fine-tuning strategies that enhance the detectability of hallucinations in the resulting model. Focusing on semantic-entropy-based detection, we observe that many erroneous outputs remain undetected because the model produces nearly identical incorrect answers across multiple runs. To address this, we propose diversity-oriented fine-tuning to encourage more varied generations. We introduce two specific strategies: one based on Supervised Fine-Tuning (SFT) and the other on Direct Preference Optimization (DPO). Extensive experiments are conducted to evaluate our approach and analyze the behavior of the models before and after fine-tuning. We find that after adopting our fine-tuning methods, the models become less likely to produce low semantic entropy responses for hallucinated answers, thereby improving the effectiveness of hallucination detection, eventually yielding results better than or comparable with state of the art methods. The code will be publicly released.

Problem

• hallucination detection
• semantic entropy
• model fine-tuning
• generation diversity
• uncertainty-based detection

Innovation

• diversity-oriented fine-tuning
• hallucination detection
• semantic entropy
• Supervised Fine-Tuning
• Direct Preference Optimization