When Replanning Becomes the Bottleneck: Budgeted Replanning for Embodied Agents

2026-08-02

AI Summary

This work addresses the challenge of context inflation and long-tail latency in embodied agents caused by frequent replanning, which undermines real-time performance. The authors propose BRACE, a novel controller integrated with the E-RECAP module, that models replanning as a budget-constrained control loop to dynamically decide whether to replan, select a replanning mode, and allocate an explicit token budget to meet latency service-level objectives (SLOs). By unifying LLM-driven replanning control, token utility prediction, cross-layer context pruning, and SLO-aware optimization, the method achieves, for the first time, per-invocation budget enforcement and cost-aware progressive token pruning. Experiments across multiple simulation platforms demonstrate 62–92% reductions in replanning tokens, SLO violation rates dropping from 85.5–100% to 4.7–50%, and an 80% success rate on high-difficulty tasks with only a 4.6% violation rate.

Abstract

Embodied agents replan frequently to recover from execution drift, partial observability, and coordination hazards, but each LLM-based replanning call can consume an accumulated textual context that grows over time and across agents. Once this context becomes large, replanning latency develops heavy tails and can miss real-time deadlines even when task success remains high, a failure mode that is hard to detect from average latency or success alone. We present BRACE, a controller that formulates replanning as a budgeted control loop by deciding whether to replan, selecting a replanning mode, and allocating an explicit token budget and latency service-level objective (SLO) while accounting for optional efficiency modules. As a reusable component, we introduce E-RECAP, a cost-aware progressive token pruning method that predicts token utility and prunes replanning contexts across transformer layers while preserving critical head and tail tokens. Across Meta Habitat, RoboFactory, and AirSim, BRACE with E-RECAP reduces replanning-call token counts by 62-92% and SLO violation rates from 85.5-100.0% to 4.7-50.0% in settings where task success is already saturated. In a harder RoboFactory setting where open-loop, frozen-plan, and No BRACE all fail, BRACE + E-RECAP reaches 80.0% success with 4.6% SLO violations, demonstrating that tail-aware per-call budgeting is effective across embodied platforms.

Problem

• replanning
• embodied agents
• latency
• token budget
• SLO violation

Innovation

• budgeted replanning
• token pruning
• embodied agents
• latency SLO
• E-RECAP