MobileWAM: Bridging World Action Models to Mobile Manipulation with Chain-of-Foresight

2026-08-05

AI Summary

Existing world action models struggle to coordinate full-body motion with large-scale dynamic environments in mobile manipulation tasks. This work proposes MobileWAM, the first world action model extended to mobile manipulation scenarios. It integrates a pretrained video diffusion Transformer with lightweight action experts, leveraging inter-layer joint attention to translate internet-scale motion priors into full-body control policies. A three-tier mixture-of-experts architecture enables soft routing to disentangle locomotion and manipulation intents, while a Chain-of-Foresight mechanism—active only during training—enhances the temporal predictive capacity of intermediate representations. MobileWAM achieves state-of-the-art performance on the ManiSkill-HAB benchmark and successfully transfers to the real-world ARX Lift2 platform, demonstrating strong multi-task generalization capabilities.

Abstract

World action models (WAMs) built on video generation backbones are a rising recipe for robot learning, yet remain confined to tabletop manipulation. Mobile manipulation demands simultaneous locomotion and whole-body manipulation amid scene-scale dynamics, yet is still dominated by dynamics-blind visual encoders with hand-crafted coordination. We bridge this gap with MobileWAM, a mixture-of-transformers architecture that fuses a pretrained video diffusion transformer with a lightweight action expert through layerwise joint attention, translating internet-scale motion priors into whole-body control. To reconcile the heterogeneous dynamics of moving and manipulating, each feed-forward layer of the action expert becomes a three-expert mixture of shared, locomotion, and manipulation experts, softly routed by the motion intent in the action tokens. To densify supervision, we further propose Chain-of-Foresight (CoF): intermediate representations sequentially predict a chain of future latent chunks, each step conditioned on its predecessor. CoF pairs naturally with our decoupled video--action denoising scheme. At deployment, the WAM serves as a pure current-frame encoder; foresight acts only through gradients, so at inference the foresight chain and video generation are discarded, leaving only policy-level cost. MobileWAM surpasses state-of-the-art mobile manipulation policies on ManiSkill-HAB and fine-tunes to a real ARX Lift2 mobile manipulator across diverse tasks with strong generalization. Code will be released soon.

Problem

• mobile manipulation
• world action models
• locomotion
• whole-body control
• scene-scale dynamics

Innovation

• Mobile Manipulation
• World Action Models
• Mixture-of-Transformers
• Chain-of-Foresight
• Video Diffusion Transformer