Technology Category
Search and Optimization: Learning to SearchReasoning under Uncertainty: Sequential Decision MakingPlanning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
Vision-driven River Following of UAV via Safe Reinforcement Learning using Semantic Dynamics Model
🤖 AI Summary
Autonomous river-following navigation for UAVs in dense river networks under GPS-denied conditions remains challenging due to limited observability and safety-critical constraints.
Method: We propose a safety-enhanced, semantics-driven reinforcement learning framework: (i) modeling river coverage control as a submodular Markov decision process; (ii) constructing an interpretable dynamics model grounded in semantic water-body masks; and (iii) designing a constrained policy architecture that jointly optimizes action selection and dynamic cost estimation, incorporating sliding-window baseline advantage estimation and Lagrangian constraint optimization.
Contribution/Results: (1) A marginal-gain advantage estimator improves policy gradient efficiency; (2) joint semantics-dynamics modeling enhances state prediction accuracy; (3) hard safety guarantees and task performance are balanced under partial observability. Simulation results demonstrate faster convergence, superior suppression of risky actions, and significantly improved reward–safety trade-off.
Technology Category
Application Category
📝 Abstract
Vision-driven autonomous river following by Unmanned Aerial Vehicles is critical for applications such as rescue, surveillance, and environmental monitoring, particularly in dense riverine environments where GPS signals are unreliable. We formalize river following as a coverage control problem in which the reward function is submodular, yielding diminishing returns as more unique river segments are visited, thereby framing the task as a Submodular Markov Decision Process. First, we introduce Marginal Gain Advantage Estimation, which refines the reward advantage function by using a sliding window baseline computed from historical episodic returns, thus aligning the advantage estimation with the agent's evolving recognition of action value in non-Markovian settings. Second, we develop a Semantic Dynamics Model based on patchified water semantic masks that provides more interpretable and data-efficient short-term prediction of future observations compared to latent vision dynamics models. Third, we present the Constrained Actor Dynamics Estimator architecture, which integrates the actor, the cost estimator, and SDM for cost advantage estimation to form a model-based SafeRL framework capable of solving partially observable Constrained Submodular Markov Decision Processes. Simulation results demonstrate that MGAE achieves faster convergence and superior performance over traditional critic-based methods like Generalized Advantage Estimation. SDM provides more accurate short-term state predictions that enable the cost estimator to better predict potential violations. Overall, CADE effectively integrates safety regulation into model-based RL, with the Lagrangian approach achieving the soft balance of reward and safety during training, while the safety layer enhances performance during inference by hard action overlay.
Problem
Research questions and friction points this paper is trying to address.
Enabling UAVs to autonomously follow rivers in GPS-denied environments
Formulating river following as a Submodular Markov Decision Process
Ensuring safety and efficiency via model-based SafeRL framework
Innovation
Methods, ideas, or system contributions that make the work stand out.
Marginal Gain Advantage Estimation for reward refinement
Semantic Dynamics Model for interpretable predictions
Constrained Actor Dynamics Estimator for SafeRL
🔎 Similar Papers
No similar papers found.
