$ω$-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation

2026-08-06

AI Summary

This work addresses the common limitation in existing approaches that decouple locomotion and manipulation when humanoid robots perform household tasks requiring simultaneous movement and interaction. The paper introduces the ω-0 model, the first latent world model for whole-body action generation tailored to real humanoid robots. Given language instructions, multi-view RGB-D observations, and proprioceptive feedback, ω-0 employs a lightweight future observation embedding and a diffusion-based generative mechanism to directly predict executable whole-body action latents. By unifying multimodal inputs into a coherent framework, the method effectively bridges human priors with robot motor execution. Evaluated across eleven diverse household tasks, ω-0 significantly outperforms strong baselines—including imitation learning, vision-language-action (VLA) models, and existing world models—demonstrating fluent and coordinated mobile manipulation behaviors.

Abstract

Humanoid household tasks often require concurrent loco-manipulation, where the robot must move, adjust posture, maintain balance, and manipulate objects as a single coordinated behavior. Yet existing humanoid policies typically decompose locomotion and manipulation, while recent world-action models remain either arm-centric or video-centered. We present $ω$-0, a latent predictive whole-body world-action model for real-world humanoid concurrent loco-manipulation. Given a language instruction, current visual observation, and robot proprioceptive state, $ω$-0 directly predicts controller-compatible whole-body action latents for real-robot execution. Rather than reconstructing future videos, $ω$-0 learns compact future observation embeddings as a lightweight predictive objective, coupling latent visual foresight with diffusion-based whole-body action generation. The model supports egocentric RGB, exocentric RGB, and exocentric depth inputs, and leverages controller-based simulation replay to ground human/public visual-motion priors into robot-executable action latents. We further collect $ω$-HOME, a 40+ hour real-world household humanoid dataset with synchronized multi-view observations, whole-body SMPL motions, robot states, and action latents. Real-world experiments on 11 household tasks demonstrate that a single $ω$-0 model can produce smooth manipulate-while-moving behaviors and consistently outperform representative imitation learning, VLA, humanoid, and WAM baselines.

Problem

• humanoid
• loco-manipulation
• world-action model
• whole-body control
• concurrent manipulation and locomotion

Innovation

• world-action model
• latent predictive modeling
• concurrent loco-manipulation
• diffusion-based action generation
• humanoid robot