A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment

2025-04-21

AI Summary

Addressing the challenges of sim-to-real transfer, stringent real-time requirements, and heavy training-data dependency when deploying deep reinforcement learning (DRL) policies for quadrotor UAVs in unstructured outdoor environments, this paper introduces AirGym—the first end-to-end, reproducible, full-stack closed-loop DRL platform. AirGym unifies high-fidelity dynamics simulation, lightweight real-time inference, a ROS 2–based embedded flight controller architecture, MAVROS middleware, and a physical experimental benchmark. It enables seamless policy development—from scratch training (e.g., PPO, SAC) and simulation validation to real-world deployment. The platform achieves robust hover, dynamic obstacle avoidance, and trajectory tracking under realistic outdoor disturbances. Multi-task policy training and deployment are compressed to minutes. All code and benchmarks are publicly released.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Deploying robot learning methods to a quadrotor in unstructured outdoor environments is an exciting task. Quadrotors operating in real-world environments by learning-based methods encounter several challenges: a large amount of simulator generated data required for training, strict demands for real-time processing onboard, and the sim-to-real gap caused by dynamic and noisy conditions. Current works have made a great breakthrough in applying learning-based methods to end-to-end control of quadrotors, but rarely mention the infrastructure system training from scratch and deploying to reality, which makes it difficult to reproduce methods and applications. To bridge this gap, we propose a platform that enables the seamless transfer of end-to-end deep reinforcement learning (DRL) policies. We integrate the training environment, flight dynamics control, DRL algorithms, the MAVROS middleware stack, and hardware into a comprehensive workflow and architecture that enables quadrotors' policies to be trained from scratch to real-world deployment in several minutes. Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments, as a physical experiment benchmark. Through extensive empirical validation, we demonstrate the efficiency of proposed sim-to-real platform, and robust outdoor flight performance under real-world perturbations. Details can be found from our website https://emnavi.tech/AirGym/.

Problem

• Bridging sim-to-real gap for quadrotor DRL deployment
• Enabling real-time onboard processing for learned policies
• Providing comprehensive training-to-deployment workflow benchmark

Innovation

• Seamless transfer of end-to-end DRL policies
• Integrated training to deployment workflow
• Rich environments for physical experiment benchmark