Rolling-WAM: World Action Models with Rolling Imagination

2026-09-24

AI Summary

This study addresses the high latency incurred by joint denoising during replanning in world action models. To mitigate this, we propose a rolling denoising strategy that introduces a novel cross-cycle progressive noise scheduling mechanism. By integrating a sliding window with diffusion models, our approach incrementally refines video-action chunks, distributing the computational load across multiple control cycles and eliminating the need to predict the future from scratch at each step. Experimental evaluations on the LIBERO benchmark and real-robot deployments demonstrate that the proposed method maintains competitive performance while achieving a 4.5× steady-state replanning speedup, substantially enhancing the real-time responsiveness of closed-loop control systems.

Abstract

World Action Models (WAMs) couple action generation with future visual prediction for robotic manipulation. However, completing the joint video-action denoising process at each replanning cycle incurs substantial latency, delaying action updates and limiting closed-loop responsiveness. We present Rolling-WAM, a formulation that distributes joint denoising across successive replanning cycles. Our method maintains a sliding window of video-action chunks at staggered noise levels. At each step, a rolling noise schedule fully denoises the imminent action chunk for execution, while partially refining farther-future chunks. As the window advances with new camera observations, the retained future chunks continue their denoising process. This distributes the computational cost over time while carrying an evolving visual-action context across chunk boundaries. Evaluations on LIBERO, RoboTwin, and a real-world Unitree G1 humanoid show that Rolling-WAM achieves competitive manipulation performance. By removing the need to denoise the entire prediction horizon from scratch, it delivers a 4.5x steady-state replanning speedup over standard joint WAMs.

Problem

• World Action Models
• Robotic Manipulation
• Replanning Latency
• Closed-loop Responsiveness
• Denoising

Innovation

• World Action Models
• Rolling Imagination
• Joint Denoising
• Sliding Window
• Robotic Manipulation