KungfuAthleteBot: learning high-dynamic humanoid motion from video with unified robust recovery

2026-10-02

AI Summary

This study addresses the challenges of physical inconsistency, absent force information, and missing recovery strategies in video data that hinder humanoid robots from learning highly dynamic motions. To overcome these limitations, this work constructs a martial arts dataset and proposes physics-guided trajectory correction to eliminate kinematic artifacts. It further introduces pseudo-low-kinetic-energy sampling initialization to avoid infeasible actions, enabling end-to-end reinforcement learning for both motion tracking and fall recovery within a unified framework. The proposed approach successfully deploys highly dynamic skills on a real-world humanoid robot, achieving autonomous recovery from arbitrary falls within approximately 0.7 seconds. This establishes a new record as the fastest single-policy fall recovery demonstrated to date.

Abstract

Video is an abundant, inexpensive source of human motion data that is rich in extreme athletic behaviors. Making it usable for humanoid robots, however, is not a matter of simply retargeting a reconstructed trajectory: video-derived motion is physically inconsistent, devoid of actuation information, and says nothing about failure or recovery. We present KungfuAthleteBot (KAB), a framework that treats learning high-dynamic motion from video as the central problem and resolves each of these three failure modes in turn. (C1) We build the KungfuAthlete dataset from videos of national-level martial artists and introduce a physics-guided parabolic trajectory correction that removes height floating, ground penetration, and high-frequency jitter from reconstructed aerial and landing phases. (C2) Because video carries no force information, strict tracking of a reconstructed trajectory is dynamically infeasible, and error-driven initialization keeps re-launching the policy from infeasible aerial poses. We introduce physics-driven pseudo-low-kinetic-energy (LKE) sampling, our central mechanism for making such references learnable: it biases initialization towards dynamically feasible states, letting the policy discover feasible actuation patterns instead of imitating infeasible ones. (C3) Finally, we introduce a direct training paradigm in which disturbance rejection and fall recovery are learned inside the same policy that tracks the video motion, requiring no recovery reference data and no manual mode switching. On a humanoid robot, KAB learns dynamic skills from video and recovers from arbitrary falls in about 0.7 s, the fastest reported recovery for a unified policy. Ablations on the unified policy confirm the necessity of its components, supporting the view that repairing and compensating video data, rather than only collecting more of it, is what unlocks high-dynamic humanoid skills.

Problem

• humanoid robot
• high-dynamic motion
• video-to-motion
• physical inconsistency
• fall recovery

Innovation

• Humanoid Motion Learning
• Physics-Guided Trajectory Correction
• Pseudo-Low-Kinetic-Energy Sampling
• Unified Fall Recovery
• Video-to-Robot Retargeting