HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document

2026-07-26

AI Summary

This work addresses the challenges of unreliable indexing and high traversal costs in graph-based approaches for retrieving and integrating cross-region, cross-modal evidence in complex document question answering. The authors propose a holistic-view-guided multimodal graph retrieval-augmented generation framework that constructs a concept-level graph structure and establishes robust cross-modal indices between this graph and multimodal segments, thereby avoiding exhaustive entity traversal. A holistic view is introduced to guide graph updates, effectively suppressing noise, while evidence is regrouped by modality to enhance fusion efficacy. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on most metrics across three benchmark datasets and significantly improves online retrieval efficiency compared to existing graph-based baselines.

Abstract

Question answering (QA) over complex documents requires models to retrieve and integrate evidence distributed across distant document regions and modalities. Multimodal GraphRAG provides a promising direction by organizing document evidence with graph structures. However, existing methods often suffer from unreliable cross-modal evidence indexing and expensive graph traversal. To address these issues, we propose HVM-GraphRAG, a holistic-view multimodal GraphRAG framework on complex document. HVM-GraphRAG uses a holistic view to guide graph construction, thereby reducing noisy and conflicting graph updates and building reliable indices between concept-level graph nodes and supporting multimodal chunks. During retrieval, HVM-GraphRAG searches over a compact concept-level graph and directly accesses supporting evidence through the constructed index, avoiding costly traversal over dense entity-level graphs. After obtaining the retrieved evidence, HVM-GraphRAG further reorganizes chunks into modality-specific groups, enabling the answering model to better integrate heterogeneous evidence. Experiments on three datasets show that HVM-GraphRAG achieves the best answer performance in most evaluated settings while substantially improving online retrieval efficiency over representative graph-based baselines.

Problem

• multimodal
• graph retrieval
• complex document
• question answering
• cross-modal indexing

Innovation

• Holistic-View
• Multimodal GraphRAG
• Concept-Level Graph
• Cross-Modal Indexing
• Retrieval Efficiency