HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models

2026-07-23

AI Summary

This work addresses the challenge of hallucinations in multimodal large language models, which often arise from inconsistencies in visual, textual, or commonsense reasoning. While existing approaches support only coarse-grained detection, this study introduces a novel fine-grained hallucination diagnosis task, unifying hallucination detection, classification, and explainable rationale generation within a tripartite framework. The authors construct HalluScope-30K, a large-scale dataset encompassing eight hallucination sources and five task scenarios, and train HalluScope-4B/8B diagnostic models using automated data generation and a multi-granularity joint reward function. Experiments demonstrate state-of-the-art performance on both the MHALO benchmark and a newly established fine-grained evaluation suite. Furthermore, the generated diagnostic feedback significantly enhances the hallucination correction capabilities of Qwen3-VL-8B-Instruct and LLaVA-1.5-7B.

Abstract

Although Multimodal Large Language Models have achieved strong performance across a wide range of vision-language tasks, they still suffer from hallucinations, where model outputs become inconsistent with the visual content, textual context, or commonsense knowledge. Existing studies primarily address this problem through coarse-grained detection. However, these approaches often provide insufficient diagnostic information for understanding hallucination types and supporting downstream hallucination mitigation. To bridge this gap, we propose fine-grained hallucination diagnosis for MLLMs, a new unified task that jointly performs hallucination detection, classification, and interpretable explanation generation. We develop an automated data generation pipeline and construct HalluScope-30K, a large-scale diagnostic dataset covering eight sources and five task categories. Based on this dataset, we design a multi-granular joint reward function and train two diagnosis models, HalluScope-4B and HalluScope-8B, which achieve state-of-the-art performance on both the MHALO benchmark and our fine-grained hallucination classification benchmark. Notably, detection and classification are mutually beneficial under joint optimization. Furthermore, diagnosis-driven feedback experiments show that the fine-grained diagnostic explanations produced by our model effectively guide target models to correct their hallucinations, with full diagnosis substantially outperforming all baselines on both Qwen3-VL-8B-Instruct and LLaVA-1.5-7B.

Problem

• hallucination
• multimodal large language models
• fine-grained diagnosis
• vision-language tasks
• model reliability

Innovation

• fine-grained hallucination diagnosis
• multimodal large language models
• hallucination classification
• interpretable explanation
• joint optimization