Actively Resolving Contextual Uncertainty for Underspecified Tasks in Natural Language

2026-09-24

AI Summary

This study addresses the challenge of high contextual uncertainty faced by robots executing linguistically ambiguous tasks in unfamiliar environments, where they must simultaneously infer success criteria and locate task-relevant information. To this end, we propose CLUE, a framework that leverages large language models to generate hypotheses and performs closed-loop verification and planning optimization through online-constructed language embedding maps, thereby actively resolving uncertainty. Our analysis demonstrates that incorporating dynamic environmental interaction significantly outperforms static language mapping approaches. Evaluated across 15 real-world tasks on a Spot robot, CLUE achieves near-optimal performance with less than a 7% gap from the optimal policy, representing a fourfold improvement over feedback-free baselines. Furthermore, it reduces vision-language model token consumption to under one-tenth of that required by conventional methods.

Abstract

Foundation models provide robots with the ability to interpret natural language and reason about environmental context, yet most language-conditioned policies assume that goals are well-specified and that task-relevant information is provided upfront via a prior map. Operating in unfamiliar environments with underspecified tasks entails high contextual uncertainty: the robot must jointly infer what constitutes task success, what constitutes relevant information, and where (or whether) that information exists. We address these limitations via CLUE (Closed-Loop contextual Uncertainty rEsolution), a framework for actively resolving contextual uncertainty given underspecified tasks in natural language. CLUE uses an LLM-derived policy to hypothesize task-relevant concepts and potential plans. It then uses a language-embedded map, which is constructed online, to ground these hypotheses into actions. The policy sequentially evaluates hypotheses via closed-loop environment interaction and refines its plans as it gathers new information. We deploy CLUE on a Boston Dynamics Spot across three real indoor and outdoor environments spanning 15 tasks that require object disambiguation, functional inference, and occlusion reasoning. CLUE achieves a success rate within 7 percentage points of an oracle policy and outperforms an LLM-enabled planner without closed-loop feedback by a 4x margin. Supporting experiments demonstrate that simply building and then querying a language-enriched map is insufficient to resolve complex contextual planning tasks; these approaches achieve roughly one third the success rate of CLUE while requiring over 10x more VLM tokens. We provide additional information at https://zacravichandran.github.io/CLUE.

Problem

• contextual uncertainty
• underspecified tasks
• natural language grounding
• robot navigation
• foundation models

Innovation

• Contextual Uncertainty
• Closed-Loop Planning
• Language-Embedded Map
• Underspecified Tasks
• Foundation Models