Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model

2026-09-23

AI Summary

研究解决了潜流世界模型中运动丢失的问题，通过解码增强滚动训练(DART)方法恢复了运动的时间结构。

Abstract

Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.

Problem

• latent world models
• frozen self-supervised latent space
• motion
• pretrained flow
• latent-only losses

Innovation

• DART
• latent world models
• motion restoration
• decode-path supervision
• frozen self-supervised latent space