PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

2026-07-30

AI Summary

This work addresses the challenge of enabling humanoid robots to achieve whole-body safe obstacle avoidance using only onboard perception. The authors propose Link-CBF, a novel approach that, for the first time, integrates perception-driven control barrier functions (CBFs) with reinforcement learning (RL). Leveraging semantic segmentation and depth images from a head-mounted camera, the method optimizes an avoidance policy through adversarial motion priors without access to privileged state information, enabling zero-shot deployment in real-world environments. Evaluated on the Unitree G1 platform, Link-CBF achieves near-optimal dodging performance in both single-throw and continuous scenarios, with a success rate of up to 95%. These results demonstrate that a fixed onboard camera suffices for effective obstacle avoidance and exhibits strong generalization across multiple ball types.

Abstract

We present PAC-MAN, a perception-aware CBF-RL framework that couples control-barrier safety with deployment-realistic onboard sensing for whole-body humanoid dodgeball. The deployed policy sees the ball only as segmentation-masked depth from a head-mounted camera, while training-time CBF guidance represents clearance to every body link, and an adversarial motion prior regularizes the resulting evasive reflexes. We evaluate on a controlled any-link contact benchmark with seeded throws in two regimes: single throws and a deployment loop in which the robot walks back to its station and recovers between throws. On this benchmark, the policy comes within a few points of a privileged state oracle: a fixed onboard camera alone is adequate for evasion. We find that usable barrier structure depends on perceptual observability: Joint-CBF gives the best performance with accurate ball states, degrades under fixed-camera observations when used only as training guidance, and recovers with a ball-tracking gimbal or privileged runtime filter. We therefore deploy a lightweight Link-CBF policy zero-shot on the Unitree G1 in the real world, where it tolerates imperfect perception, succeeds on 95% of throws, and uses semantic segmentation to dodge different balls.

Problem

• humanoid dodgeball
• whole-body safety
• onboard perception
• control barrier functions
• real-world deployment

Innovation

• Perception-Aware CBF
• Whole-Body Safety
• Humanoid Dodgeball
• Onboard Sensing
• Zero-Shot Deployment