Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robots

2025-09-08

AI Summary

Sim-to-real transfer for legged robots suffers from insufficient motion robustness and energy efficiency, alongside heavy reliance on manual reward tuning. Method: We propose a systematic transfer framework that eliminates the need for dynamic parameter randomization. Our approach integrates a first-principles-based permanent magnet synchronous motor (PMSM) energy consumption model with deep reinforcement learning to design a compact, physics-driven reward function comprising four terms—unifiedly characterizing electrical and mechanical energy losses. Combined with bottom-up dynamical parameter identification, it enables high-fidelity alignment of energy behavior between simulation and reality. Contribution/Results: Evaluated on 13 heterogeneous legged robots, our method reduces the whole-body transport cost of ANYmal by 32% to 1.27—surpassing state-of-the-art approaches. It is the first to achieve robust locomotion while ensuring interpretability, minimal hyperparameter dependence, and high energy efficiency in policy transfer.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Legged robots must achieve both robust locomotion and energy efficiency to be practical in real-world environments. Yet controllers trained in simulation often fail to transfer reliably, and most existing approaches neglect actuator-specific energy losses or depend on complex, hand-tuned reward formulations. We propose a framework that integrates sim-to-real reinforcement learning with a physics-grounded energy model for permanent magnet synchronous motors. The framework requires a minimal parameter set to capture the simulation-to-reality gap and employs a compact four-term reward with a first-principle-based energetic loss formulation that balances electrical and mechanical dissipation. We evaluate and validate the approach through a bottom-up dynamic parameter identification study, spanning actuators, full-robot in-air trajectories and on-ground locomotion. The framework is tested on three primary platforms and deployed on ten additional robots, demonstrating reliable policy transfer without randomization of dynamic parameters. Our method improves energetic efficiency over state-of-the-art methods, achieving a 32 percent reduction in the full Cost of Transport of ANYmal (value 1.27). All code, models, and datasets will be released.

Problem

• Bridging sim-to-real transfer gap for legged robots
• Addressing energy efficiency neglect in existing controllers
• Overcoming complex reward formulations with compact solution

Innovation

• Sim-to-real reinforcement learning with energy model
• Compact four-term reward with energetic loss formulation
• Minimal parameter set for reliable policy transfer