Smoothness as a Constraint for Stable Humanoid Locomotion

2026-09-21

AI Summary

本文针对人形机器人全身控制策略中平滑性与任务响应的平衡问题，提出了一种名为DeCap的新约束强化学习算法，通过将身体分为上半身和下半身两部分来分别施加平滑性约束，从而在保持稳定性的同时提高动作效率。

Abstract

Embodied AI systems, particularly humanoid robots deployed in real world scenarios require whole-body control policies that are both task-responsive and physically smooth. However, smoothness is not uniform across the body: lower body must remain sufficiently reactive, while the upper body must be tightly regulated to preserve stability. Existing reinforcement learning approaches typically impose smoothness through auxiliary terms in the reward function, which compete with task objectives, treating the body as uniform and provide no direct control over the physical quantities responsible for smooth behavior. We introduce DeCap (Decoupled Constraint-aware policy), a constrained reinforcement learning algorithm that decouples whole-body smoothness into separate upper- and lower-body constraint groups, each formulates smoothness as explicit constraints on physical motion limits. To improve constraint satisfaction near feasibility boundaries, DeCap incorporates a bounded barrier penalty that activates proactively as limits are approached and remains bounded at the constraint limit. On real-world humanoid whole-body control task, DeCap reduces upper-body action rate by 2.50x and acceleration by 2.18x relative to reward-based smoothness policies, while also improving lower-body smoothness and reducing transient motion. We demonstrate that a fixed set of smoothness constraints transfers across diverse terrains, alleviating the need of extensive reward tuning.

Problem

• Smoothness
• Humanoid Locomotion
• Whole-body Control
• Reinforcement Learning
• Constraint

Innovation

• DeCap
• Constrained Reinforcement Learning
• Smoothness Constraints
• Bounded Barrier Penalty
• Whole-Body Control