DataScout: Automatic Data Fact Retrieval for Statement Augmentation with an LLM-Based Agent

2025-04-24

AI Summary

Multi-perspective fact retrieval in data storytelling is time-consuming, heavily reliant on manual analysis, and struggles to balance objectivity with efficiency. Method: This paper proposes a user-agent collaborative, stance-aware retrieval tree framework. It integrates large language model–based reasoning agents, stance-driven query strategies, and a retrieval tree generation algorithm, mapping outputs to an interactive mind-map visualization interface. Contribution/Results: The framework introduces the first collaborative retrieval architecture featuring user controllability, agent interpretability, and stance traceability—enabling dynamic user intervention, full process transparency, and comprehensive multi-dimensional fact coverage. Case studies and expert evaluations demonstrate that the system reduces average retrieval time by 47%, significantly improves claim verification efficiency and narrative credibility, and diminishes dependence on domain-specific analytical expertise of content creators.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Preferences

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

A data story typically integrates data facts from multiple perspectives and stances to construct a comprehensive and objective narrative. However, retrieving these facts demands time for data search and challenges the creator's analytical skills. In this work, we introduce DataScout, an interactive system that automatically performs reasoning and stance-based data facts retrieval to augment the user's statement. Particularly, DataScout leverages an LLM-based agent to construct a retrieval tree, enabling collaborative control of its expansion between users and the agent. The interface visualizes the retrieval tree as a mind map that eases users to intuitively steer the retrieval direction and effectively engage in reasoning and analysis. We evaluate the proposed system through case studies and in-depth expert interviews. Our evaluation demonstrates that DataScout can effectively retrieve multifaceted data facts from different stances, helping users verify their statements and enhance the credibility of their stories.

Problem

• Automates data fact retrieval for comprehensive narratives
• Reduces time and skill barriers in data analysis
• Enhances statement credibility with multifaceted data facts

Innovation

• LLM-based agent for data fact retrieval
• Interactive retrieval tree for collaborative control
• Mind map visualization for intuitive steering