Structured Uncertainty guided Clarification for LLM Agents

2025-11-11

AI Summary

To address tool invocation errors in LLM agents caused by ambiguous user instructions, this paper proposes an active clarification framework grounded in structured uncertainty. Methodologically, it introduces the first formal modeling of structured uncertainty over tool parameters; formulates multi-turn clarification as a cost-constrained partially observable Markov decision process (POMDP), optimizing question selection via expected value of perfect information (EVPI); incorporates aspect-level cost modeling to suppress redundant queries; and employs uncertainty-weighted GRPO for reinforcement learning training. Contributions include: (1) ClarifyBench—the first benchmark supporting multi-domain dialogue clarification; (2) improvements in task coverage by 7–39% across domains and a 1.5–2.7× reduction in clarification turns; and (3) a significant increase in When2Call accuracy from 36.5% to 65.2% on a 3B model, substantially outperforming all baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

LLM agents extend large language models with tool-calling capabilities, but ambiguous user instructions often lead to incorrect invocations and task failures. We introduce a principled formulation of structured uncertainty over tool-call parameters, modeling joint tool-argument clarification as a POMDP with Expected Value of Perfect Information (EVPI) objective for optimal question selection and aspect-based cost modeling to prevent redundancy. Our SAGE-Agent leverages this structured uncertainty to achieve superior efficiency: increasing coverage on ambiguous tasks by 7-39% while reducing clarification questions by 1.5-2.7$ imes$ compared to strong prompting and uncertainty-based baselines. We present ClarifyBench, the first multi-turn tool-augmented disambiguation benchmark with realistic LLM-based user simulation across diverse domains including document editing, vehicle control, and travel booking. Additionally, we demonstrate that structured uncertainty provides effective training signals for reinforcement learning, boosting When2Call accuracy from 36.5% to 65.2% (3B model) and 36.7% to 62.9% (7B model) through uncertainty-weighted GRPO training. These results establish structured uncertainty as a principled, efficient approach for tool-augmented agents, improving both task success and interaction efficiency in real-world scenarios.

Problem

• LLM agents face ambiguous user instructions causing incorrect tool invocations
• Existing methods lack structured uncertainty modeling for tool-call parameters
• Current approaches struggle with inefficient clarification questioning during interactions

Innovation

• Modeling tool-call uncertainty as POMDP with EVPI
• Using aspect-based cost modeling to reduce redundancy
• Leveraging uncertainty for reinforcement learning training signals