LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts

2026-08-01

AI Summary

This work addresses the significant performance gap between FastSAC and PPO in humanoid full-body motion tracking, where FastSAC achieves high training efficiency but substantially lower tracking accuracy. To bridge this gap, the authors propose LooperMuscle, a novel framework that integrates a structured mixture-of-experts (MoE) policy, an expert-aware distributed critic, contribution-based routing for experience replay, and a delayed curriculum scheduler. These components jointly establish a closed-loop training mechanism that coordinates expert specialization, value learning, and data routing. Remarkably, LooperMuscle surpasses FastSAC’s tracking accuracy within approximately 45 minutes of training and approaches the performance of PPO—a method typically requiring six hours of training—thereby enabling both highly efficient and high-fidelity motion control policy learning.

Abstract

FastSAC-style methods significantly reduce humanoid motion training time but often suffer from notable performance degradation compared with PPO in whole-body tracking tasks. We target this speed-performance gap by introducing LooperMuscle, a composed expert policy learning framework that restores tracking quality while preserving high training efficiency. LooperMuscle combines a semantically structured mixture-of-experts actor, an expert-aware distributional critic, and contribution-routed replay with deferred curriculum scheduling. These three components form a closed training loop in which expert contributions guide data routing, routed data shape value learning, and value gradients in turn refine expert specialization. Empirically, our approach substantially outperforms vanilla FastSAC in motion tracking accuracy while requiring far less wall-clock time than PPO: where FastSAC trains in about 15 minutes but underperforms, and PPO achieves stronger results but requires about 6 hours, LooperMuscle recovers a substantial fraction of the remaining gap to PPO in roughly 45 minutes of simulation training, delivering practical efficiency for rapid policy iteration. The code will be released to benefit the research community at https://loopermuscle.github.io/.

Problem

• humanoid whole-body tracking
• training efficiency
• performance degradation
• speed-performance trade-off
• motion tracking accuracy

Innovation

• Mixture-of-Experts
• Whole-body tracking
• Fast reinforcement learning
• Expert-aware critic
• Curriculum scheduling