Learning Reusable Hybrid Motion Priors for Humanoid Locomotion from Motion Imitation

2026-07-27

AI Summary

This work addresses the limitations of traditional reinforcement learning in policy reuse and the restricted applicability of motion imitation to fixed trajectories. The authors propose a three-stage framework: first, an expert policy is trained to imitate human motion; second, policy distillation yields a frozen Hybrid Motion Prior (HMP) comprising a proprioceptive encoder, a residual vector quantization (RVQ) codebook, and an action decoder; third, downstream tasks reuse the HMP by selecting discrete codebook entries. The key innovations include the first formulation of imitated skills as a shareable, frozen prior, the interpretability of RVQ codebook entries—such as modulating gait via activated layers—and the introduction of rotation-based techniques to refine latent space structure and reduce falls. Experiments demonstrate that HMP significantly enhances training efficiency and stability in speed tracking, navigation, and fall recovery tasks, with successful real-world deployment on the Unitree G1 robot.

Abstract

Reinforcement learning can produce robust humanoid controllers, but each new task is typically trained as a separate policy with its own reward design and training process. Motion imitation provides an alternative source of motor competence by training policies to track retargeted human motions, yet the resulting controllers remain reference trackers and are not directly usable as task policies. We propose a three-stage pipeline that turns motion-imitation skills into a reusable hybrid motion prior (HMP) for humanoid locomotion. First, an expert policy is trained to imitate retargeted human motion-capture clips. Second, the expert is distilled into a frozen architecture composed of a proprioceptive encoder, a residual vector-quantized (RVQ) codebook, and an action decoder. Third, task-level policies are trained to solve locomotion tasks by selecting discrete codebook entries while the HMP remains frozen. We evaluate the method on velocity tracking, point-goal navigation, and fall-recovery velocity tracking in simulation, and deploy the velocity-tracking policy on a real Unitree G1 robot. The distillation process preserves the tracking behavior of the expert, while the resulting HMP can be reused without retraining as the action interface for different downstream locomotion policies. The learned HMP reveals an interpretable codebook structure in which the number of active RVQ stages modulates the available gait patterns. We further show that training the codebook with the rotation trick improves latent organization and reduces downstream falls compared with a standard straight-through estimator.

Problem

• humanoid locomotion
• motion imitation
• reusable policy
• hybrid motion prior
• reinforcement learning

Innovation

• hybrid motion prior
• motion imitation
• vector-quantized codebook
• policy distillation
• humanoid locomotion