Metro-WM: Long-Horizon Latent Planning with Realisable Sub-Goals

2026-09-25

AI Summary

This study addresses the failure of long-horizon planning in hierarchical frameworks caused by macro-level predictors generating physically infeasible subgoals. To overcome this, we propose Metro-WM, a framework that introduces a novel graph-planning mechanism based on empirical frame retrieval. Specifically, Metro-WM employs a joint embedding architecture to extract real states from offline expert demonstrations, constructing a graph structure for path search rather than relying on unconstrained latent vector generation. This approach fundamentally ensures the physical feasibility and execution robustness of subgoals. Experimental results demonstrate that Metro-WM improves success rates on long-horizon tasks by 37.33% and accelerates planning speed by 10.9×, significantly reducing computational overhead while discovering shorter paths than those provided in the demonstrations.

Abstract

Model-predictive control with Joint-Embedding Predictive Architectures (JEPAs) provides a strong zero-shot goal-reaching planner, but it is only effective over short planning horizons. Hierarchical extensions attempt to bridge this gap by learning a macro planner to predict intermediate latent sub-goals to guide the micro planner. In this work, we demonstrate that unconstrained latent sub-goal prediction is fundamentally flawed. A rigorous evaluation reveals that a leading state-of-the-art macro planner routinely emits physically unrealisable sub-goals. To resolve this, we introduce Metro-WM, a hierarchical framework that issues sub-goals by retrieving genuine states from prior experience rather than generating ungrounded latent vectors. Specifically, Metro-WM constructs a graph whose vertices are observed frames from offline expert demonstrations or random-action trajectories, allowing frames from different episodes to be connected and stitched into routes to the goal. Planning over the full graph also makes the system highly robust to execution errors: if the micro planner drifts off course, Metro-WM instantly finds a new optimal path from the current state. Our experiments show that Metro-WM achieves superior long-horizon success rates of up to 37.33 percentage points over the next best hierarchical approach while being up to 10.9 times faster, requiring both 13-56 times less offline compute and fewer tuned hyperparameters. Additional analysis reveals that Metro-WM finds shorter paths than the offline demonstrations, outperforms an oracle relying on the query's own demonstration, and maintains robust performance under extremely sparse dataset conditions.

Problem

• long-horizon planning
• latent sub-goals
• hierarchical planning
• model-predictive control
• unrealisable sub-goals

Innovation

• Hierarchical Planning
• Joint-Embedding Predictive Architectures
• Graph-based Sub-goal Retrieval
• Long-Horizon Control
• Model-Predictive Control