Structured LLM Reasoning for Zero-Shot Human--Robot Coordination Under Hidden Goals

2026-08-04

AI Summary

This work addresses the challenge of achieving zero-shot, efficient coordination in human-robot collaboration when agents’ goals are partially unobservable. The authors propose a structured approach grounded in the Dec-POMDP framework that, for the first time, integrates large language models with theory-of-mind reasoning. Their method employs five core modules—hierarchical planning, semantic parsing of dialogue, action feasibility verification, explainable inference of implicit goals, and feedback-driven replanning—to generate physically feasible and interpretable collaborative behaviors. Real-world human-robot experiments demonstrate that the proposed approach significantly reduces interaction steps compared to an ablated model without theory-of-mind capabilities and an offline multi-agent reinforcement learning baseline, while also substantially improving post-task human trust ratings.

Abstract

We present a structured large-language-model (LLM) architecture for zero-shot human--robot coordination in a cooperative construction task with private goal views. Guided by a Dec-POMDP formulation, the architecture decomposes decision-making into (i) action-conditioned Theory-of-Mind (ToM) inference, (ii) hierarchical planning, (iii) conversation interpretation, (iv) action verification, and (v) feedback-based replanning. We compare the proposed method with an ablation without ToM inference and a multi-agent reinforcement-learning policy trained offline over many goal pairs. In human-participant experiments, the proposed method required fewer interaction steps and yielded higher post-interaction trust ratings than both baselines. These results suggest that systematically decomposing the team decision problem, using LLMs as tractable surrogates for otherwise intractable inference and planning computations, and retaining conventional verification for physical feasibility can improve both task coordination and the human experience.

Problem

• zero-shot coordination
• hidden goals
• human-robot collaboration
• Dec-POMDP
• Theory-of-Mind

Innovation

• Structured LLM Reasoning
• Zero-Shot Coordination
• Theory-of-Mind Inference
• Dec-POMDP
• Human-Robot Collaboration