Anticipatory Risk-Guided Reinforcement Learning for Safe Flight Through Dynamic Clutter

2026-07-26

AI Summary

This work addresses the challenge of safe navigation for quadrotors in dynamic, cluttered environments, where perception latency and the absence of reliable spatiotemporal features hinder robust operation. To this end, we propose a forward-looking, risk-aware reinforcement learning framework that leverages privileged simulator states to construct a direction-aligned future collision risk map based on the Closest Point of Approach (CPA). An asymmetric Actor-Critic architecture trains a policy network to predict structured risk solely from onboard depth sequences. Integrating a physics-informed risk prediction mechanism with a lightweight spatiotemporal encoder, our method extracts motion cues directly from depth sequences without requiring explicit object tracking or optical flow estimation. The resulting end-to-end visual policy enables efficient zero-shot sim-to-real transfer, significantly enhancing flight safety and navigation efficiency in both simulated and real-world environments.

Abstract

Safe quadrotor navigation in cluttered and dynamic environments depends not only on instantaneous geometric perception, but more critically on anticipating collision risks induced by relative motion. Conventional modular pipelines frequently suffer from perception latency, while end-to-end learning methods relying on implicit scalar rewards often struggle to extract reliable spatio-temporal features without physics-grounded supervision. To address this, we propose an anticipatory risk-guided reinforcement learning framework. Leveraging privileged simulator states, we construct a directionally aligned future collision risk map based on the Closest Point of Approach (CPA). Through an asymmetric actor-critic architecture, the network is trained to self-predict this structured risk, which explicitly guides the visual policy during deployment. A lightweight spatio-temporal encoder extracts motion cues directly from onboard depth sequences, bypassing explicit object tracking or optical flow estimation. Extensive simulated and real-world experiments demonstrate that our method effectively improves safety margins and flight efficiency in dense dynamic clutters compared to existing baselines. Furthermore, the learned policy achieves robust zero-shot Sim-to-Real transfer on a physical quadrotor, relying purely on abstracted spatio-temporal depth sequences and its self-predicted risk priors, validating the effectiveness of our approach and its robust generalization from simulation to reality.

Problem

• safe navigation
• dynamic clutter
• collision risk anticipation
• quadrotor
• spatio-temporal perception

Innovation

• anticipatory risk
• reinforcement learning
• Closest Point of Approach (CPA)
• spatio-temporal encoding
• Sim-to-Real transfer