YAC: Bridging Natural Language and Interactive Visual Exploration with Generative AI for Biomedical Data Discovery

2025-09-23

AI Summary

This study addresses the fragmentation between natural language (NL) querying and interactive visualization in biomedical data discovery. We propose a hybrid interaction paradigm that tightly couples NL understanding with interactive visual analytics. Our approach employs a multi-agent system integrating large language models (LLMs), declarative output generation, and programmable frontend widgets to enable an end-to-end closed loop—from NL input to structured responses, synchronized chart rendering, and dynamic data filtering. A key innovation is the introduction of user-adjustable semantic controls, enabling fine-grained, iterative refinement of generated outputs. Evaluation across four representative biomedical use cases demonstrates significant improvements: +28.6% in query response accuracy, −32% in cognitive workload (measured by NASA-TLX), and −41% in task completion time. The framework establishes a novel, explainable, AI-augmented paradigm for human-in-the-loop data discovery.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Incorporating natural language input has the potential to improve the capabilities of biomedical data discovery interfaces. However, user interface elements and visualizations are still powerful tools for interacting with data, even in the new world of generative AI. In our prototype system, YAC, Yet Another Chatbot, we bridge the gap between natural language and interactive visualizations by generating structured declarative output with a multi-agent system and interpreting that output to render linked interactive visualizations and apply data filters. Furthermore, we include widgets, which allow users to adjust the values of that structured output through user interface elements. We reflect on the capabilities and design of this system with an analysis of its technical dimensions and illustrate the capabilities through four usage scenarios.

Problem

• Bridging natural language and interactive visualizations for biomedical data discovery
• Generating structured declarative output using a multi-agent system approach
• Allowing users to adjust structured output values through interface widgets

Innovation

• Generates structured declarative output via multi-agent system
• Interprets output to render linked interactive visualizations
• Provides widgets for adjusting structured output values