WB-WAM: Heterogeneous Body-Hand Pre-training for Humanoid Loco-Manipulation

2026-09-28

AI Summary

This study addresses the insufficient pretraining data coverage and the difficulty of jointly learning body and hand motions in whole-body coordinated manipulation for humanoid robots. To this end, it proposes a world action model incorporating whole-body motion supervision. Methodologically, heterogeneous data are integrated within a unified physical action space to enable joint video-action pretraining. This work pioneers the introduction of explicit whole-body motion supervision into generative video pretraining, further combining PICO mid-training optimization priors with auxiliary forward kinematics supervision to substantially reduce reliance on real-world demonstration data. Experimental results demonstrate that the proposed model achieves an 81.9% success rate in the HumanoidArena simulation and an average success rate of 84.0% across five real-world tasks, validating its highly effective sim-to-real data transfer capability.

Abstract

Humanoid loco-manipulation demands coordinated body and hand behavior, while conventional robot pre-training data provide limited coverage of such whole-body motion. We present WB-WAM, a World Action Model that incorporates explicit whole-body action supervision into generative video pre-training. A shared physical action space integrates body, root, and dexterous hand annotations from heterogeneous sources, enabling joint video and action learning from 1880.2 hours of partially annotated video and motion data. The resulting priors are refined through PICO mid-training and adapted to robot tasks with auxiliary forward kinematics supervision. We construct WB-Datasets to support these stages with retargeted egocentric human demonstrations and robot trajectories, allowing task-aligned human motion to supplement limited robot data. Evaluations in simulation demonstrate strong whole-body task performance with 81.9% in HumanoidArena, while real-world experiments further validate WB-WAM with 84.0% mean success across five tasks. Moreover, task-aligned PICO mid-training improves downstream task performance while reducing the need for real-robot demonstrations. These results support heterogeneous whole-body pre-training and human motion transfer as a practical route to data-efficient humanoid loco-manipulation.

Problem

• Humanoid loco-manipulation
• Whole-body motion
• Pre-training data
• Body-hand coordination

Innovation

• World Action Model
• Heterogeneous Pre-training
• Humanoid Loco-manipulation
• Human Motion Transfer
• PICO Mid-training