From Failures to Supervision: DynamicEnvPlan for Robust Long-Horizon Embodied Planning

2026-08-01

AI Summary

This work addresses the challenge of planning failures in long-horizon embodied agents operating in dynamic environments, where environmental changes and execution uncertainties often lead to task failure. To tackle this issue, the authors propose DynamicEnvPlan, a closed-loop framework that explicitly converts execution failures into supervisory signals. By generating controlled perturbations and applying constrained corrections, the method constructs recovery trajectories that enable adaptive replanning. The approach integrates structured semantic memory, high-level skills, and staged supervised fine-tuning to facilitate robust dynamic recovery. Evaluated across 104 task-scenario combinations, DynamicEnvPlan improves success rates from 33.3% to 76.2% and consistently outperforms existing baselines across seven critical metrics, including safety and functional compliance.

Abstract

Physical-world interaction is inherently dynamic, as environments can evolve during execution, requiring agents to adapt their plans under non-stationary conditions. We study this challenge through long-horizon embodied planning under environment deviations and execution uncertainty. Existing embodied-task benchmarks can expose such failures, but these failures are usually treated as evaluation outcomes instead of learnable signals for training agents to recover. In this work, we introduce DynamicEnvPlan, a closed-loop framework for high-level planning in dynamic environments. It extends embodied task execution with humanoid agents, high-level primitive skills, structured semantic memory, and controllable perturbations. Our data synthesis design consists of planning, perturbation, and guarded correction modules that turn dynamic execution states into recovery-oriented traces. The resulting traces are used for staged supervised fine-tuning, enabling the planner to learn from both nominal execution and perturbed recovery trajectories. Using 104 task-scene combinations spanning i.i.d., compositional generalization, and out-of-distribution settings for fine-tuning and evaluation, DynamicEnvPlan boosts success rate from 33.3% for the base planner to 76.2%, while improving across all seven evaluation metrics critical to physical-world interaction, including safety and affordance compliance.

Problem

• dynamic environments
• long-horizon planning
• embodied agents
• execution uncertainty
• failure recovery

Innovation

• DynamicEnvPlan
• embodied planning
• recovery-oriented learning
• structured semantic memory
• supervised fine-tuning