VisPile: A Visual Analytics System for Analyzing Multiple Text Documents With Large Language Models and Knowledge Graphs

2025-10-10

AI Summary

To address low efficiency in multi-document sensemaking and insufficient credibility of AI-generated outputs in intelligence analysis, this paper proposes VisPile—a visual collaborative analysis framework integrating large language models (LLMs) and knowledge graphs (KGs). VisPile tightly couples LLMs’ semantic comprehension with KGs’ structured reasoning within an interactive visual workflow, supporting document clustering, abstractive summarization, relational pattern mining, and dynamic evidence validation and refinement. Its key innovations include a bidirectional LLM–KG enhancement mechanism and an explainable, human-controllable human–AI collaboration design. Evaluated by six domain-expert intelligence analysts, VisPile significantly improves multi-document integration efficiency and analytical insight quality, demonstrating practical utility and trustworthiness in real-world intelligence analysis scenarios.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Data Mining & Knowledge Management: Intelligent Query Processing
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Intelligence analysts perform sensemaking over collections of documents using various visual and analytic techniques to gain insights from large amounts of text. As data scales grow, our work explores how to leverage two AI technologies, large language models (LLMs) and knowledge graphs (KGs), in a visual text analysis tool, enhancing sensemaking and helping analysts keep pace. Collaborating with intelligence community experts, we developed a visual analytics system called VisPile. VisPile integrates an LLM and a KG into various UI functions that assist analysts in grouping documents into piles, performing sensemaking tasks like summarization and relationship mapping on piles, and validating LLM- and KG-generated evidence. Our paper describes the tool, as well as feedback received from six professional intelligence analysts that used VisPile to analyze a text document corpus.

Problem

• Enhancing sensemaking in large document collections
• Integrating LLMs and knowledge graphs for text analysis
• Assisting analysts in document grouping and evidence validation

Innovation

• Integrates large language models with knowledge graphs
• Enables document grouping and relationship mapping
• Provides validation for AI-generated evidence