DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models

2026-07-28

AI Summary

Existing world-action models (WAMs) rely on photorealistic video reconstruction, making them susceptible to irrelevant visual cues such as texture and illumination, which introduces computational redundancy and limits generalization. This work proposes a dynamic egocentric perspective that emphasizes the temporal dynamics of the gripper, object, and contact regions, steering the model to attend to action-relevant visual changes rather than appearance reconstruction. Through a dynamic egocentric supervision mechanism and DynaRoute—a dynamic correlation routing module—combined with temporal difference stream matching, trajectory-guided weighting, and token-based dynamic attention bias, the approach shifts the focus of visual learning toward interaction-induced dynamics without increasing deployment complexity. Experiments demonstrate that the method significantly improves policy performance in both simulated and real-world robotic manipulation tasks and exhibits enhanced robustness under out-of-distribution perturbations in lighting, appearance, and background.

Abstract

World-Action Models (WAMs) augment robot policies with future visual prediction, but it remains unclear what the visual modality should learn for control. While photorealistic future prediction provides dense supervision, it also incurs substantial computation and can allocate capacity to texture, illumination, and background variations that are only weakly related to action selection. Recent efficient WAM variants suggest that the main benefit of the video branch may not lie in the rendered future itself, but in the control-relevant visual representations induced during training. In this work, we revisit future video prediction from a dynamic-centric perspective and ask whether an existing RGB-based WAM can be redirected from appearance-dominated reconstruction toward interaction-induced visual dynamics without introducing additional modality-specific predictions or online inputs at deployment. We propose DC-WAM, a dynamic-centric WAM framework that redistributes supervision and computation in the RGB video branch. At the supervision level, DC-WAM combines temporal-difference flow matching with trajectory-guided weighting, emphasizing dense temporal changes and localized regions where the gripper, manipulated objects, and contact areas move. At the reasoning level, DynaRoute predicts token-wise dynamic relevance and converts it into an attention bias, guiding the model toward control-relevant future tokens. Experiments in simulation and on real-world manipulation tasks show that DC-WAM consistently improves policy performance, especially under out-of-distribution perturbations in lighting, object appearance, and background texture.

Problem

• World-Action Models
• visual supervision
• dynamic-centric
• future prediction
• control-relevant representations

Innovation

• dynamic-centric learning
• visual dynamics
• flow matching
• attention bias
• World-Action Models