DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulation

2026-08-01

AI Summary

This work addresses the limitations of existing world-action models in dynamic manipulation, which rely solely on the current frame and execute backbone networks synchronously, thereby struggling to perceive object motion and respond in real time. The paper proposes DynamicWAM, the first framework to incorporate a dual-path motion-conditioning mechanism: one path encodes historical optical flow using a frozen pretrained video VAE to preserve spatial motion structure, while the other injects explicit kinematic descriptors—such as displacement, velocity, and acceleration—into action experts. These streams are fused via joint world-action attention. Combined with model distillation and a real-time chunking (RTC) asynchronous execution strategy, DynamicWAM significantly enhances responsiveness and control accuracy, achieving a 38.2% success rate (53.2 manipulation score) on the DOMINO benchmark and an average success rate of 46.7% across 12 real-world tasks, outperforming the strongest baseline by 22.9 percentage points.

Abstract

Dynamic manipulation requires robots to infer target motion and respond promptly, yet existing World-Action Models (WAMs) typically condition only on the current frame and execute large backbones synchronously, limiting motion awareness and responsive control in dynamic scenes. We propose DynamicWAM, a compact WAM for dynamic object manipulation with dual-path motion conditioning. DynamicWAM introduces history-flow conditioning, encoding temporally aligned optical-flow frames alongside the current observation through a frozen pretrained video VAE to preserve spatial motion structure, while injecting kinematic descriptors of displacement, duration, velocity, and acceleration into the action expert to provide motion magnitude and timing. The two complementary paths are fused through joint world-action attention. A distilled compact backbone and real-time chunking (RTC)-based asynchronous execution further enable responsive control. On DOMINO, DynamicWAM achieves a 38.2% success rate and a 53.2 manipulation score, outperforming all evaluated baselines. Across 12 real-world tasks spanning linear, circular, and compound target motion, it achieves a 46.7% average success rate, exceeding the strongest baseline by 22.9 percentage points.

Problem

• Dynamic manipulation
• World-Action Models
• Motion awareness
• Responsive control
• Temporal conditioning

Innovation

• dual-path motion conditioning
• history-flow conditioning
• kinematic descriptors
• world-action attention
• asynchronous execution