MotionJEPA: Preventing Temporal Feature Collapse by Capturing Visual Changes in Latent Space

2026-09-20

AI Summary

本文针对JEPA训练中特征抑制和潜在表示崩溃问题，提出了一种名为DISReg的新正则化方法，并将其整合进MotionJEPA架构，以促进静态和动态特征的平衡学习。

Abstract

Joint Embedding Predictive Architectures (JEPAs) are a promising paradigm for learning task-agnostic latent world models without visual reconstruction. However, standard JEPA training exhibits a strong inductive bias towards slow features, causing feature suppression and the collapse of latent representation. While inverse dynamics provides temporal anti-collapse, it relies on action labels and offers little incentive to embed general, unlabeled dynamics. We introduce Difference Image and Single image embedding Regularization (DISReg), a novel regularizer that builds on an inverse-dynamics-style module that predicts temporal difference image embeddings without any pixel reconstruction loss, encouraging balanced static and dynamic feature learning. DISReg consists of a static term that shapes the distribution of the image embedding and encourages slow features, and a dynamic term, which, unlike direct regularization on the embedding, imposes no constraint on the image embedding's shape or distribution and instead only incentivizes that dynamic features be present. By integrating this regularizer into a standard JEPA, we establish our new architecture, MotionJEPA. Latent probing demonstrates that MotionJEPA produces more complete representations than other methods, and our trajectory analysis shows it maintains geometrically simple latent embeddings with low curvature. We further show that MotionJEPA improves downstream planning success under static-background distractors across four environments.

Problem

• Joint Embedding Predictive Architectures
• temporal feature collapse
• slow features

Innovation

• Difference Image and Single image embedding Regularization
• DISReg
• temporal difference image embeddings
• balanced static and dynamic feature learning
• MotionJEPA