LadderMan: Learning Humanoid Perceptive Ladder Climbing

2026-06-04

AI Summary

This work addresses the challenge of robust ladder climbing for humanoid robots under conditions of sparse support, complex whole-body coordination, and high sensitivity in perception and control. The authors propose a two-stage learning framework that integrates hybrid motion tracking with imitation-reinforcement learning and leverages a vision foundation model to bridge the sim-to-real gap in depth perception, enabling zero-shot deployment in real-world environments. Through a dual-agent teleoperation architecture, the system achieves robust climbing across diverse ladder configurations and successfully executes a range of manipulation tasks in constrained settings, significantly enhancing the generalization and dexterous capabilities of humanoids in demanding vertical scenarios.

Abstract

Humanoid robots hold great promise for operating in human-centered environments, yet ladder climbing remains one of the most challenging tasks due to sparse footholds and handholds, complex whole-body coordination, and sensitivity to perception and control errors. We present \textbf{LadderMan}, a unified system that enables humanoid robots to robustly climb diverse ladders and perform manipulation under such constrained conditions. Our climbing policy is built on a scalable two-stage learning pipeline, where we use hybrid motion tracking to learn multiple climbing experts from a single reference motion, and distill these experts into a unified depth-based visuomotor climbing policy via hybrid imitation and reinforcement learning. To enable real-world deployment, we leverage vision foundation models to bridge the sim-to-real gap in depth perception. Building on the learned climbing policy, we further train a separate manipulation policy using a dual-agent formulation, allowing stable on-ladder manipulation via teleoperation. Experiments demonstrate that LadderMan achieves robust ladder climbing across a wide range of geometries, successfully transfers to real-world hardware in a zero-shot manner, and supports various manipulation tasks under challenging ladder constraints. Video results are available at https://ladderman-robot.github.io .

Problem

• humanoid robots
• ladder climbing
• perception
• whole-body coordination
• control errors

Innovation

• humanoid robotics
• visuomotor policy
• sim-to-real transfer
• imitation learning
• vision foundation models