RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments

2026-09-30

AI Summary

This study addresses the challenge of balancing long-term performance with probabilistic safety constraints in perception-based navigation within unknown environments. To this end, we propose a safe navigation framework that integrates reinforcement learning (RL) with stochastic nonlinear model predictive control. Specifically, the method constructs a probabilistic actor-critic architecture based on probably approximately correct (PAC) sampling, leveraging finite-time statistical guarantees to achieve safe exploration under hard constraints while approximating the long-term optimal RL policy subject to probabilistic safety requirements. This approach significantly enhances navigation safety for high-dimensional complex systems. Furthermore, its superior performance is validated through real-world flight experiments on fixed-wing unmanned aerial vehicles.

Abstract

In this paper, we present an approach for combining stochastic nonlinear model predictive control (SNMPC) and reinforcement learning (RL) to enable probabilistically-safe perception-based navigation in unknown environments. Our method first uses RL to train probabilistic actor-critic and sensor prediction models. We then leverage these probabilistic models in a sampling-based SNMPC framework known as Probably Approximately Correct (PAC)-NMPC, which uses hard constraints to enforce finite-time statistical guarantees on the probability of collision and value function improvement. By ensuring that our finite-horizon SNMPC policies decrease the value function in expectation, we can approach the long-horizon performance of the RL approach while satisfying probabilistic safety constraints. Through simulation experiments, we show that our approach can improve the safety of perception-based RL navigation policies and scale to high dimensional systems with large sensor input spaces and complex nonlinear dynamics. We also demonstrate our approach through hardware experiments, showing improved performance for vision-based navigation with an agile fixed-wing aerial vehicle in unknown environments.

Problem

• perception-based navigation
• probabilistic safety
• unknown environments
• nonlinear model predictive control
• reinforcement learning

Innovation

• Reinforcement Learning
• Stochastic Nonlinear Model Predictive Control
• PAC-NMPC
• Probabilistic Safety
• Perception-Based Navigation