Learning Expressive and Compositional Motion Representation via Spectral Skills

2026-09-29

AI Summary

This study addresses the challenges of low action prediction accuracy and difficult behavior composition in humanoid robot planning and control by proposing a spectral skill latent representation method. Unlike conventional reconstruction-based approaches, this method employs predictive representation learning to encode short motion segments through future trajectory forecasting. Integrated with a hierarchical architecture and a language-conditioned planner, it enables seamless concatenation of independent skills and the synthesis of novel behaviors via superposition along orthogonal directions. Experimental results demonstrate that the proposed approach reduces global tracking error by 62%. Furthermore, precise tracking, skill chaining, and behavior composition capabilities are successfully validated on the Unitree G1 humanoid robot platform.

Abstract

Robotic foundation models offer a promising path toward general-purpose humanoid robot control, often through hierarchical architectures. However, their effectiveness depends on the command interface between the planner and the controller, which must support accurate execution while remaining easy to predict, and ideally allow new behaviors to be composed from prior ones. In this work, we introduce spectral skills, a latent representation of this interface that meets these requirements through predictive representation learning. By design, spectral skills compactly encode short motion segments and are learned by predicting subsequent motion rather than reconstructing the encoder input. On a 29-DoF humanoid, a controller conditioned on spectral skills reduces global tracking error by 62\% relative to the state of the art. The same frozen controller chains independently encoded skills without a separate transition policy. It also composes new behaviors by adding orthogonal directions to any compatible base skill, producing combinations unseen in the training data. We demonstrate tracking, chaining, and composition, as well as control through a language-conditioned planner, on Unitree G1 hardware. Project page: https://spectral-skill.github.io

Problem

• humanoid robot control
• foundation models
• command interface
• motion representation
• behavior composition

Innovation

• Spectral Skills
• Predictive Representation Learning
• Motion Composition
• Humanoid Control
• Skill Chaining