XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation

2025-06-10

AI Summary

GraphRAG systems suffer from poor debuggability and interpretability due to heavy reliance on large language model (LLM) calls during graph construction and querying, coupled with opaque, black-box pipelines—hindering effectiveness evaluation and failure diagnosis. To address this, we propose the first end-to-end visual analytics paradigm for GraphRAG, covering the full pipeline: graph construction → subgraph retrieval → LLM-based generation. Our interactive web framework integrates synchronized multi-view visualization, query-path highlighting, subgraph provenance mapping, and fine-grained correlation of LLM invocation logs. This enables comprehensive traceability across all stages. Empirical evaluation demonstrates a 3.2× acceleration in fault attribution and a 41% improvement in key recall identification accuracy. The open-source system has been widely adopted by the research and practitioner community, supporting GraphRAG optimization in multiple real-world applications.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Graph-based Retrieval-Augmented Generation (RAG) has shown great capability in enhancing Large Language Model (LLM)'s answer with an external knowledge base. Compared to traditional RAG, it introduces a graph as an intermediate representation to capture better structured relational knowledge in the corpus, elevating the precision and comprehensiveness of generation results. However, developers usually face challenges in analyzing the effectiveness of GraphRAG on their dataset due to GraphRAG's complex information processing pipeline and the overwhelming amount of LLM invocations involved during graph construction and query, which limits GraphRAG interpretability and accessibility. This research proposes a visual analysis framework that helps RAG developers identify critical recalls of GraphRAG and trace these recalls through the GraphRAG pipeline. Based on this framework, we develop XGraphRAG, a prototype system incorporating a set of interactive visualizations to facilitate users' analysis process, boosting failure cases collection and improvement opportunities identification. Our evaluation demonstrates the effectiveness and usability of our approach. Our work is open-sourced and available at https://github.com/Gk0Wk/XGraphRAG.

Problem

• Analyzing GraphRAG effectiveness on datasets
• Understanding complex GraphRAG information processing pipeline
• Improving GraphRAG interpretability and accessibility

Innovation

• Graph-based RAG enhances LLM with structured knowledge
• Visual analysis framework improves GraphRAG interpretability
• Interactive visualizations aid failure analysis and improvements