MEGRAG: Multi-Granular Evidence Graphs for Answer-Aware Multi-Hop RAG

2026-08-03

AI Summary

This work addresses the challenge in multi-hop question answering where evidence is scattered and existing iterative retrieval-augmented generation (RAG) methods suffer from reliance on a single granularity of supporting information and error propagation, hindering effective reasoning. To overcome these limitations, the authors propose an answer-aware, multi-granularity evidence graph framework that constructs a cross-granularity index—spanning passages, sentences, and knowledge triples—offline. During inference, retrieval starts from compact triples and dynamically incorporates contextual information, with intermediate answers guiding decisions on whether to terminate or generate focused queries for further retrieval. This approach enables dynamic granularity selection and early answer verification, thereby reducing redundant retrieval and mitigating error accumulation. Experimental results demonstrate that the proposed framework significantly improves both accuracy and efficiency over multiple RAG baselines on multi-hop QA benchmarks.

Abstract

Multi-hop question answering is a fundamental challenge in retrieval-augmented generation (RAG), because deriving an answer requires integrating dispersed evidence. Iterative RAG (iRAG) is widely used for this challenge, but existing methods have two limitations. First, most methods still support each reasoning step with single-granularity evidence, making it difficult to balance information density and contextual noise. Second, existing methods often answer the original question only after aggregating evidence retrieved across intermediate steps, so redundant evidence and intermediate retrieval errors may accumulate and degrade the final answer. To address these limitations, we propose MEGRAG, an answer-aware framework that represents multi-hop reasoning as a path-structured multi-granular evidence graph. Offline, MEGRAG links passages to their sentences and extracted triples through a cross-granularity index. Online, it retrieves passages for the current query and selects aligned evidence, starting with compact triples and adding sentence or passage context as needed. MEGRAG uses the resulting intermediate answer and prior reasoning to decide whether the Initial Query has been resolved. If not, it identifies the missing information and formulates a focused next query; otherwise, it stops retrieval and returns the answer. Extensive experiments demonstrate consistent gains over a diverse set of RAG baselines.

Problem

• multi-hop question answering
• retrieval-augmented generation
• evidence granularity
• reasoning error accumulation
• answer-aware retrieval

Innovation

• multi-granular evidence
• answer-aware retrieval
• multi-hop reasoning
• evidence graph
• RAG