Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models

2026-08-04

AI Summary

Existing world action models struggle to balance efficiency and robustness during inference: retaining future-aware representations incurs high computational costs, while more efficient approaches compromise temporal generalization. This work proposes a sparse future conditioning mechanism that leverages a SparseMoT module and an Interval KV-Fusion strategy to selectively reuse multi-depth future representations without increasing attention complexity, thereby avoiding redundant video-action interactions. Seamlessly integrated into the diffusion-based action denoising pipeline, the method boosts success rates on LIBERO-Plus from 49.14% to 73.57%, significantly outperforming Fast-WAM, and achieves a 2.21× faster inference speed than Joint-WAM. It further attains state-of-the-art performance across LIBERO, RoboTwin 2.0, and real-world robotic tasks.

Abstract

World Action Models (WAMs) improve robot manipulation by learning how the environment evolves beyond the current observation. However, existing approaches face a fundamental dilemma: Joint-WAMs preserve future-aware representations during inference but incur prohibitive computation costs, while efficient alternatives remove future modeling at inference time and may lose the robustness benefits of temporal reasoning. In this work, we revisit the role of future representations in WAMs and show that inference-time future conditioning is critical for generalization under distribution shifts. This observation motivates Faster-WAM, an efficient future-conditioning WAM that preserves future representations while avoiding expensive video-action interaction. Faster-WAM introduces a sparse future-conditioning framework that computes future representations once and selectively reuses them throughout action denoising. Specifically, we propose SparseMoT to replace ubiquitous layer-wise fusion with selective video-action interaction at a compact subset of network stages, and Interval KV-Fusion to aggregate multi-depth future representations without increasing attention complexity. Experiments demonstrate that Faster-WAM achieves a substantially better performance-efficiency trade-off than existing WAMs. On the out-of-distribution LIBERO-Plus benchmark, Faster-WAM improves success rate from 49.14% to 73.57% compared with Fast-WAM, while running 2.21$\times$ faster than Joint-WAM. It further achieves state-of-the-art performance on LIBERO and RoboTwin 2.0, while demonstrating strong robustness in real-world manipulation.

Problem

• World Action Models
• future conditioning
• inference efficiency
• temporal reasoning
• distribution shift

Innovation

• future conditioning
• sparse interaction
• efficient inference
• world action models
• temporal reasoning