Disentangling Ambiguity from Instability in Large Language Models: A Clinical Text-to-SQL Case Study

2026-02-12

AI Summary

This work addresses the challenge of output diversity in clinical Text-to-SQL tasks, where variability may stem from either input ambiguity or model instability—factors that must be disentangled to enable targeted interventions. The authors propose CLUES, a novel framework that models Text-to-SQL as a two-stage “explanation → answer” process. By leveraging a bipartite semantic graph and Schur complement decomposition, CLUES is the first method to explicitly decouple and quantify these two sources of uncertainty in clinical settings. Beyond offering an actionable diagnostic mechanism, the approach significantly enhances human review efficiency: just 25% of queries identified as high-ambiguity or high-instability account for 51% of all errors. Empirical results demonstrate that CLUES outperforms Kernel Language Entropy in failure prediction across multiple benchmarks.

Technology Category

Machine Learning: Calibration & Uncertainty QuantificationReasoning under Uncertainty: Other Foundations of Reasoning under UncertaintyKnowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsSearch and Retrieval-Augmented AI: Web query analysis, representation and understandingWeb Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

Deploying large language models for clinical Text-to-SQL requires distinguishing two qualitatively different causes of output diversity: (i) input ambiguity that should trigger clarification, and (ii) model instability that should trigger human review. We propose CLUES, a framework that models Text-to-SQL as a two-stage process (interpretations -->answers) and decomposes semantic uncertainty into an ambiguity score and an instability score. The instability score is computed via the Schur complement of a bipartite semantic graph matrix. Across AmbigQA/SituatedQA (gold interpretations) and a clinical Text-to-SQL benchmark (known interpretations), CLUES improves failure prediction over state-of-the-art Kernel Language Entropy. In deployment settings, it remains competitive while providing a diagnostic decomposition unavailable from a single score. The resulting uncertainty regimes map to targeted interventions - query refinement for ambiguity, model improvement for instability. The high-ambiguity/high-instability regime contains 51% of errors while covering 25% of queries, enabling efficient triage.

Problem

• ambiguity
• instability
• Text-to-SQL
• large language models
• clinical text

Innovation

• disentanglement
• semantic uncertainty
• Schur complement
• Text-to-SQL
• model instability