Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking

2026-06-06

AI Summary

This work addresses the challenge of unstable or unsafe navigation in humanoid robots operating in complex dynamic environments, often caused by imprecise foothold control. The authors propose a lightweight and general reinforcement learning framework that employs a dynamic goal sampler to generate foothold supports and trains a terrain-agnostic 3D foothold tracking policy as a low-level controller. The approach leverages a goal representation robust to noise in pose and contact estimation, eliminating reliance on idealized state assumptions or task-specific pipelines, thereby significantly enhancing policy generality and real-world transferability. Experimental results demonstrate that the framework achieves accurate and natural foothold control in both simulation and physical platforms, and seamlessly integrates with diverse high-level planners, providing a reliable foundation for humanoid loco-manipulation tasks.

Abstract

Enabling humanoid robots to operate in complex, dynamic environments remains a critical challenge, fundamentally limited by the ability to navigate robustly, safely, and accurately. While reinforcement learning with velocity-commanded policies has achieved remarkable robustness in humanoid locomotion, this approach lacks explicit control of the foothold placement, leading to unsafe behavior, such as stepping onto human feet, or imprecise navigation, hindering the following manipulation task. Conversely, explicit foothold-tracking policies offer a promising alternative by directly being commanded with target foot poses. However, existing approaches are often limited by unrealistic state assumptions, compromising real-world deployment, or they are part of staged pipelines, making them tied to specific downstream tasks. In this work, we introduce a novel, lightweight framework for training general-purpose 3D foothold-tracking policies. By dynamically providing footstep support through a goal sampler, this method enables the learned policy to be agnostic to specific terrains. Our new target representation effectively mitigates challenges arising in the real world, such as noisy and inaccurate pose estimation and foot contact estimation. Designed for direct real-world transfer, our policy acts as a standalone low-level controller that can be seamlessly paired with various high-level foothold generators. We demonstrate the effectiveness of our framework through extensive experiments in simulation and in the real world. By coupling our policy with different upstream planners, we achieve natural and accurate locomotion in challenging settings, paving the way for loco-manipulation tasks in complex environments.

Problem

• humanoid locomotion
• foothold tracking
• reinforcement learning
• real-world deployment
• loco-manipulation

Innovation

• foothold tracking
• humanoid locomotion
• reinforcement learning
• real-world transfer
• general-purpose policy