Ambiguity Resolution in Text-to-Structured Data Mapping

2025-05-16

AI Summary

Severe lexical and semantic ambiguity in natural language significantly impairs the accuracy of large language models (LLMs) in text-to-structured-data mapping tasks—e.g., tool calling and text-to-SQL. Existing approaches rely on reactive, trial-and-error reasoning (e.g., ReACT) or supervised fine-tuning, both of which suffer from low efficiency and heavy dependence on annotated data. Method: We propose an unsupervised, pre-emptive ambiguity detection framework that integrates sparse autoencoders (SAEs) with path-kernel integrated gradients to quantify a novel “concept-missing distance”—a geometric measure of representation shift in latent space induced by semantic incompleteness. This enables explicit, interpretable identification of ambiguous inputs without task-specific supervision or model fine-tuning. Contribution/Results: Our method achieves significant accuracy gains in ambiguous tool-calling tasks, eliminates inefficient exploratory decoding, and establishes a new paradigm for robust, interpretable structured mapping—fully unsupervised and annotation-free.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Ambiguity in natural language is a significant obstacle for achieving accurate text to structured data mapping through large language models (LLMs), which affects the performance of tasks such as mapping text to agentic tool calling and text-to-SQL queries. Existing methods of ambiguity handling either exploit ReACT framework to produce the correct mapping through trial and error, or supervised fine tuning to guide models to produce a biased mapping to improve certain tasks. In this paper, we adopt a different approach that characterizes the representation difference of ambiguous text in the latent space and leverage the difference to identify ambiguity before mapping them to structured data. To detect ambiguity of a sentence, we focused on the relationship between ambiguous questions and their interpretations and what cause the LLM ignore multiple interpretations. Different to the distance calculated by dense embedding vectors, we utilize the observation that ambiguity is caused by concept missing in latent space of LLM to design a new distance measurement, computed through the path kernel by the integral of gradient values for each concepts from sparse-autoencoder (SAE) under each state. We identify patterns to distinguish ambiguous questions with this measurement. Based on our observation, We propose a new framework to improve the performance of LLMs on ambiguous agentic tool calling through missing concepts prediction.

Problem

• Resolving ambiguity in text-to-structured data mapping
• Identifying ambiguity causes in LLM latent space
• Improving ambiguous agentic tool calling performance

Innovation

• Characterize ambiguity via latent space representation differences
• Design new distance measurement using path kernel and SAE
• Improve tool calling via missing concepts prediction framework