Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain

2026-06-06

AI Summary

Existing foundation models for humanoid robot behaviors often fail to reliably execute human-like motions due to their neglect of terrain variations. This work proposes a terrain-aware control framework that adaptively adjusts contact points, postures, and timing based on local terrain observations to translate human motion priors into robot-executable commands. The core innovations include a Terrain-Conforming Reference Synthesis (TCRS) method and an identity-gated Transformer tracker, which preserve the original motion semantics while introducing terrain-specific corrections only when necessary. By integrating contact-aware foothold generation, foot geometry optimization, support-aware root reconstruction, and multi-contact inverse kinematics, the framework enables terrain-conforming behavior transfer from a blind-trained teacher model to a student model, allowing the robot to accurately reproduce diverse human motions on complex terrains.

Abstract

Humanoid behavior foundation models aim to acquire reusable whole-body control policies from broad human motion priors, enabling a single controller to produce diverse and expressive behaviors. However, existing motion-centric foundation policies largely assume that the reference motion is already physically compatible with the robot's surroundings. This assumption breaks when the demonstrator, operator, and robot inhabit different environments: a human motion may specify the intended behavior, but not the footholds, clearance, body height, or contact timing required by the robot's local terrain. We introduce \emph{Perceptive Behavior Foundation Model} (Perceptive BFM), a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception. The model preserves raw kinematic motion references as the behavioral interface, while using local terrain observations to adapt contacts, posture, and timing. To provide scalable terrain supervision, we develop \emph{terrain-conformal reference synthesis} (TCRS), which converts locomotion-oriented human motion clips into terrain-consistent references through contact-aware foothold construction, foot-geometry-aware swing optimization, support-aware root reconstruction, collision repair, and multi-point inverse kinematics. We then train a blind adapted-reference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student through target-frame action alignment. The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways initialized to preserve the motion-tracking prior and trained to produce local corrections only when needed.

Problem

• humanoid robots
• motion priors
• terrain adaptation
• behavior foundation model
• perceptive control

Innovation

• Perceptive Behavior Foundation Model
• terrain-aware control
• humanoid locomotion
• reference synthesis
• Transformer-based tracking