A Generative AI System for Biomedical Data Discovery with Grammar-Based Visualizations

2025-09-19

AI Summary

To address the disconnect between natural language interaction and visualization specification generation in biomedical data exploration, this paper proposes a multi-agent collaborative framework integrating generative AI with grammar-based visualization (e.g., Vega-Lite). The framework employs specialized agents to jointly parse user intent, generate executable visualization specifications, dynamically apply semantic filters, and construct an interactive dashboard supporting real-time refinement. Its key contributions are: (1) the first deep integration of large language model–driven NL2Vis capabilities with grammar-based visualization compilation, enabling an end-to-end, editable, and verifiable pipeline from natural language queries to validated visualization components; and (2) preservation of traditional UI controllability and provenance. Evaluated on biomedical use cases, the prototype system demonstrates significant improvements in analytical efficiency and exploratory flexibility.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Humans and AI: Intelligent User Interfaces

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

We explore the potential for combining generative AI with grammar-based visualizations for biomedical data discovery. In our prototype, we use a multi-agent system to generate visualization specifications and apply filters. These visualizations are linked together, resulting in an interactive dashboard that is progressively constructed. Our system leverages the strengths of natural language while maintaining the utility of traditional user interfaces. Furthermore, we utilize generated interactive widgets enabling user adjustment. Finally, we demonstrate the potential utility of this system for biomedical data discovery with a case study.

Problem

• Combining generative AI with grammar-based visualizations for biomedical data discovery
• Creating interactive dashboards through multi-agent visualization specification generation
• Enabling user-adjustable biomedical data exploration via generated interactive widgets

Innovation

• Combines generative AI with grammar-based visualizations
• Uses a multi-agent system to generate specifications
• Creates an interactive dashboard with linked visualizations