Fixing Imbalanced Attention to Mitigate In-Context Hallucination of Large Vision-Language Model

2025-01-21

AI Summary

Large vision-language models (LVLMs) frequently generate hallucinated content in image captioning—describing elements absent from the input image—primarily due to deep visual anchoring degradation, where cross-modal alignment weakens in deeper transformer layers. Method: We propose a training-free attention rectification framework: (i) a novel dual-stream visual token selection mechanism that jointly incorporates local detail fidelity and spatial saliency; and (ii) a head-specific attention modulation strategy that dynamically enhances visual grounding in a layer-wise and sensitivity-aware manner. Contribution/Results: Our method significantly reduces hallucination rates by 62.3% on MSCOCO while preserving original task performance (e.g., CIDEr, BLEU). It is the first work to demonstrate that inference-time attention intervention alone can substantially improve LVLMs’ visual faithfulness—without architectural modification or retraining—establishing a lightweight, plug-and-play paradigm for hallucination mitigation.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities in understanding and describing visual content, achieving state-of-the-art performance across various vision-language tasks. However, these models frequently exhibit hallucination behavior, where they generate descriptions containing objects or details absent in the input image. Our work investigates this phenomenon by analyzing attention patterns across transformer layers and heads, revealing that hallucinations often stem from progressive degradation of visual grounding in deeper layers. We propose a novel attention modification approach that combines selective token emphasis and head-specific modulation to maintain visual grounding throughout the generation process. Our method introduces two key components: (1) a dual-stream token selection mechanism that identifies and prioritizes both locally informative and spatially significant visual tokens, and (2) an attention head-specific modulation strategy that differentially amplifies visual information processing based on measured visual sensitivity of individual attention heads. Through extensive experimentation on the MSCOCO dataset, we demonstrate that our approach reduces hallucination rates by up to 62.3% compared to baseline models while maintaining comparable task performance. Our analysis reveals that selectively modulating tokens across attention heads with varying levels of visual sensitivity can significantly improve visual grounding without requiring model retraining.

Problem

• Large Models
• Visual Language Tasks
• Hallucination Errors

Innovation

• Dual-Channel Selection Strategy
• Attention Adjustment
• Error Reduction in LVLMs