Learning Impact-Rich Rotational Maneuvers via Centroidal Velocity Rewards and Sim-to-Real Techniques: A One-Leg Hopper Flip Case Study

2025-05-18

AI Summary

Dynamic rotational maneuvers—such as full forward somersaults—pose significant challenges for reinforcement learning due to large angular momentum generation and high-impact ground forces, leading to training instability and sim-to-real transfer failure. To address this, we propose: (1) a global reward function based on center-of-mass angular velocity, replacing joint-level rewards prone to local optima; (2) a dynamics-aware constraint mechanism integrating motor workspace modeling and transmission load regularization to enhance robustness under extreme operational conditions; and (3) a hybrid framework combining PPO-based policy optimization with domain adaptation techniques. We demonstrate, for the first time on a monopedal jumping robot, successful hardware deployment of a complete, stable full forward somersault—including controlled takeoff, mid-air rotation, and reliable landing. This validates the efficacy and transfer reliability of our approach in high-rotation, high-impact scenarios, establishing a scalable sim-to-real paradigm for highly dynamic locomotion control.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Dynamic rotational maneuvers, such as front flips, inherently involve large angular momentum generation and intense impact forces, presenting major challenges for reinforcement learning and sim-to-real transfer. In this work, we propose a general framework for learning and deploying impact-rich, rotation-intensive behaviors through centroidal velocity-based rewards and actuator-aware sim-to-real techniques. We identify that conventional link-level reward formulations fail to induce true whole-body rotation and introduce a centroidal angular velocity reward that accurately captures system-wide rotational dynamics. To bridge the sim-to-real gap under extreme conditions, we model motor operating regions (MOR) and apply transmission load regularization to ensure realistic torque commands and mechanical robustness. Using the one-leg hopper front flip as a representative case study, we demonstrate the first successful hardware realization of a full front flip. Our results highlight that incorporating centroidal dynamics and actuator constraints is critical for reliably executing highly dynamic motions.

Problem

• Learning dynamic rotational maneuvers with large angular momentum
• Overcoming sim-to-real transfer challenges for impact-rich behaviors
• Ensuring realistic torque commands under extreme conditions

Innovation

• Centroidal velocity rewards for rotational dynamics
• Actuator-aware sim-to-real transfer techniques
• Motor operating regions modeling for robustness