FF-JEPA: Long-Horizon Planning in World Models with Latent Planners

2026-06-08

AI Summary

This work addresses the limitations of existing JEPA-based world models, which suffer from high computational costs in long-horizon planning and reliance on goal-conditioning images that hinder generalization to real-world scenarios. The authors propose FF-JEPA, a hierarchical dual forward-dynamics architecture comprising an action-conditional forward model and a latent planner that operates without explicit actions. The latent planner decomposes long-horizon tasks into short-horizon optimization problems by predicting subgoals, thereby introducing—for the first time—a goal-image-free latent planning mechanism. By integrating forward-forward learning with joint embedding prediction, FF-JEPA effectively mitigates long-horizon collapse. Experiments on the PushT benchmark demonstrate that the method significantly improves long-horizon planning performance, achieving efficient and successful planning without requiring goal references.

Abstract

Joint Embedding Predictive Architectures (JEPAs) have shown promising world modeling capabilities, enabling planning in latent space by optimizing action trajectories using methods like the Cross-Entropy Method (CEM). These methods are, however, too computationally expensive and ineffective for long-horizon planning. Furthermore, these methods typically require an explicit image of the goal state, which is not always possible in real-world tasks. In this work, we tackle these limitations by proposing Forward-Forward-JEPA (FF-JEPA), a hierarchical approach leveraging two forward dynamics models. Alongside a standard action-conditioned forward model, we introduce an action-free latent planner that predicts the next subgoal given the current state. This approach removes the need for goal images and enables long-horizon planning by decomposing complex trajectories into a sequence of tractable, short-term optimization problems. Preliminary results on PushT demonstrate that FF-JEPA successfully overcomes flat world models' long-horizon collapse, highlighting this approach as a promising direction for goal-free planning.

Problem

• long-horizon planning
• world models
• goal-free planning
• latent planners
• computational efficiency

Innovation

• FF-JEPA
• latent planner
• long-horizon planning
• goal-free planning
• hierarchical world models