WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon

2026-09-28

AI Summary

This study addresses the challenges of modeling heterogeneous controls and balancing long-horizon consistency with real-time responsiveness in interactive world models by proposing the WorldPlay2 framework. Methodologically, it introduces a factorized hybrid control interface to decouple heterogeneous actions and incorporates shared compressed memory tokens to enhance long-range dependency modeling. Furthermore, a Stable Forcing distillation strategy is developed based on an autoregressive student and bidirectional teacher architecture, combined with few-step initialization and full-trajectory replay to optimize training. Experimental results demonstrate that this approach significantly reduces distillation overhead while outperforming existing baselines in generalization capability and generation quality, effectively achieving a unified balance between real-time responsiveness and long-horizon consistency.

Abstract

Interactive world models require responding in real time to versatile controls and maintaining long-horizon consistency. However, modeling heterogeneous controls remains difficult, while explosive contexts and unstable distillation impede achieving both long-horizon consistency and real-time responsiveness. In this paper, we present WorldPlay2, an interactive world model that couples a factorized hybrid control interface with a co-design of compressed memory and stable distillation. 1) Our factorized hybrid control interface integrates frame-aligned action control with structured semantic control that explicitly disentangles scene appearance, character identity, and dynamic semantic events, thereby facilitating effective control learning. 2) To achieve efficient long-horizon modeling, we compress historical contexts into compact memory tokens shared by the autoregressive student and the bidirectional teacher. This design enables clip-wise, memory-conditioned score evaluation instead of jointly processing an entire long rollout, substantially reducing distillation overhead. 3) We further propose Stable Forcing, which initializes the autoregressive student via a few-step strategy and leverages full-rollout replay to preserve the quality of long-horizon rollouts, ensuring robust and stable distillation. Extensive experiments demonstrate the strong generalizability of our model and its superior performance compared to existing methods.

Problem

• Interactive World Models
• Heterogeneous Controls
• Long-horizon Consistency
• Real-time Responsiveness
• Knowledge Distillation

Innovation

• Interactive World Models
• Factorized Hybrid Control
• Compressed Memory
• Stable Forcing
• Knowledge Distillation