Praxis: Distilling Physical Interaction Priors from Egocentric Videos for Generalizable Whole-Body Manipulation

2026-09-24

AI Summary

This study addresses the challenge of balancing workspace reachability and hand-level interaction precision in whole-body manipulation for mobile humanoid robots under data-limited conditions. To this end, this work proposes a novel framework that distills physical interaction priors from a single human demonstration. By integrating first-person video guidance, closed-loop pose calibration, and visuo-tactile online perception, the approach synergizes navigation, alignment, and dexterous manipulation to achieve synchronous upper- and lower-body control. Notably, the method enables zero-shot skill generalization and perturbation recovery without retraining. Its spatial, visual, and cross-object generalization capabilities are validated across five long-horizon tasks, demonstrating robust resistance to external physical disturbances.

Abstract

Mobile humanoid manipulation requires both reaching a usable workspace and preserving precise hand-object interactions as object poses and contact conditions change. Learning these behaviors from limited task-specific data remains challenging. To bridge this gap, we introduce Praxis, a whole-body manipulation framework that combines physical interaction priors from one-shot egocentric video demonstrations with closed-loop posture calibration and online perception. The framework coordinates three stages: vision-language-guided navigation toward target objects, closed-loop posture calibration to align the arm-hand workspace, and dexterous manipulation with synchronized upper- and lower-body control. Online visual feedback re-grounds demonstrated interaction geometry under new object poses and scene configurations, while tactile feedback adapts hand motions to actual contact conditions. Each manipulation skill is specified by one human demonstration, without task-specific manipulation-policy retraining. Experiments across five long-horizon manipulation tasks demonstrate spatial, visual, and cross-object generalization, as well as recovery from external physical disturbances across all three stages.

Problem

• mobile humanoid manipulation
• whole-body control
• hand-object interaction
• limited demonstration data
• generalization

Innovation

• Whole-Body Manipulation
• Egocentric Video
• Physical Interaction Priors
• One-Shot Demonstration
• Closed-Loop Posture Calibration