SANE Schema-aware Natural-language Evaluation of Biological Data

2026-06-03

AI Summary

This work addresses the challenge of reliably querying structured biological data generated by high-throughput microscopy using natural language, a task hindered by the hallucination tendencies of large language models (LLMs). To this end, the authors propose SANE—a training-free text-to-SQL paradigm that integrates schema-aware prompting with constraint-based mechanisms. The study introduces a domain-specific, schema-aware evaluation framework, featuring automatically generated benchmarks and query guardrails to enable scalable, systematic, and reproducible assessment. Experimental results demonstrate that, within well-defined constrained schemas, few-shot LLMs augmented with structured prompts substantially improve query accuracy. Moreover, the analysis reveals that input ambiguity—not SQL generation errors—is the primary source of failure, highlighting the critical role of precise natural language specification in reliable database interaction.

Abstract

High-throughput microscopy generates large, structured datasets capturing cellular responses to pharmacological perturbations, but accessing these datasets typically requires SQL expertise. Large language models offer a natural-language alternative, yet their tendency to hallucinate raises concerns about result reliability . We present SANE Schema-Aware Natural-language Evaluation, a novel paradigm for domain-specific text-to-SQL evaluation: schema-grounded, automatically generated benchmarks tied to real and specific experimental structure. SANE makes evaluation more scalable, systematic, and reproducible. Using SANE, we evaluate a few-shot large language model and show that, under constrained schemas with structured prompting and guardrails, accurate query generation is achievable without any model training or fine-tuning. Most failures stem from ambiguous or underspecified inputs and manifest as overly cautious clarification requests or answers to queries that should first be disambiguated, rather than incorrect SQL generation. These results indicate that few-shot large language models can provide reliable database access in well-defined domains when combined with schema-aware prompting.

Problem

• natural-language querying
• text-to-SQL
• biological data
• large language models
• hallucination

Innovation

• schema-aware evaluation
• text-to-SQL
• large language models
• few-shot learning
• biological data querying