KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation

2026-08-04

AI Summary

This work addresses the lack of a unified benchmark for evaluating knowledge-related hallucinations in multimodal large language models across four key dimensions: entities, attributes, relations, and factual knowledge. To this end, the authors propose KnowHal, the first multimodal evaluation framework that systematically incorporates knowledge hallucination assessment. KnowHal enables controlled cross-dimensional comparison by constructing paired positive and negative questions grounded in shared images and entities, and further introduces robustness testing against false premises. The benchmark is built through a semi-automated pipeline combining large language model generation, CLIP-based semantic filtering, and human verification, yielding 1,800 samples spanning 50 categories across 10 domains. Experiments on 14 state-of-the-art models reveal that the knowledge dimension poses the greatest challenge and that model performance significantly degrades on negative samples, exposing their vulnerability to false premises.

Abstract

Hallucination remains a critical challenge for developing trustworthy Multimodal Large Language Models (MLLMs). While existing benchmarks mainly focus on entity, attribute, and relation hallucinations, knowledge-related failures are often investigated separately, lacking a unified evaluation framework across different hallucination dimensions. To overcome this, we propose \textbf{KnowHal}, a benchmark that explicitly incorporates knowledge hallucination into multimodal hallucination evaluation spanning four dimensions: entity, attribute, relation, and knowledge. KnowHal constructs paired positive and negative questions over shared images and entities, enabling controlled comparisons among perceptual errors, knowledge-related errors, and false-premise acceptance. The benchmark contains 1,800 samples across 10 domains and 50 categories, constructed through a semi-automated pipeline combining LLM assistance, CLIP-based filtering, and human verification. We evaluate 14 representative MLLMs on KnowHal and conduct extensive analyses. Results show that the knowledge dimension consistently presents the greatest challenge for nearly all evaluated models, while most models exhibit substantial performance degradation on negative questions, revealing limited robustness to false premises. By unifying four hallucination dimensions with paired question design, KnowHal addresses an important gap in existing evaluation frameworks and enables a more comprehensive assessment of hallucinations in MLLMs.

Problem

• multimodal hallucination
• knowledge hallucination
• evaluation benchmark
• false premise
• trustworthy MLLMs

Innovation

• multimodal hallucination
• knowledge hallucination
• paired question design
• benchmark evaluation
• MLLM robustness