Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation

2026-06-02

AI Summary

This work proposes AgenticRL, a novel framework that integrates multimodal generative agents into the reinforcement learning loop to overcome the limitations of handcrafted reward functions and laborious hyperparameter tuning in traditional deep reinforcement learning. By interpreting natural language instructions and visual observations, the framework autonomously generates reward functions, trains policies, and enables joint self-optimization of both rewards and policies through diagnostic feedback. Leveraging semantic-driven scene understanding and policy selection, AgenticRL combines PPO, vision-conditioned navigation, and sim-to-real transfer to achieve a 91% success rate in real-world drone navigation tasks, a 71% improvement in policy performance over initial rewards, and a 94% sim-to-real fidelity.

Abstract

Deep reinforcement learning has shown strong potential for enabling autonomous robots to learn complex navigational tasks. However, its practical use still depends heavily on human designed reward functions and repeated manual fine tuning, which is time consuming and does not guarantee high success in the desired task. This paper presents AgenticRL, agent guided reinforcement learning framework that increases autonomy in reward design, policy refinement, and real world deployment for unmanned aerial vehicles (UAV) navigation tasks. AgenticRL uses a multimodal generative pre-trained tansformer (GPT) agent to interpret task information and visual scene observations, generate task specific reward functions, train policies using Proximal Policy Optimization (PPO) algorithm, and then act as a critic by evaluating the trained policy through diagnosis packets to generate feedback. Based on this feedback, the agent identifies failure modes and refines the reward function in a closed loop self improvement process. To further leverage the multimodal GPT agent during inference, AgenticRL uses real world images and natural language task information to automatically identify the active scenario and select the appropriate trained policy for execution. The framework is evaluated on multiple navigational tasks, including gate traversal, obstacle avoidance, wall barrier crossing with landing, trajectory following, and motion behavior learning. Experimental results show that the closed loop refinement process improves policy behavior compared with initial rewards by 71%. We also demonstrate sim-to-real transfer of the proposed framework, achieving a real world success rate of 91% and a sim-to-real accuracy of 94%.

Problem

• reward design
• UAV navigation
• autonomous policy refinement
• sim-to-real transfer
• vision-conditioned reinforcement learning

Innovation

• Agentic Reinforcement Learning
• Multimodal GPT Agent
• Self-Refining Reward Design
• Vision-Conditioned UAV Navigation
• Sim-to-Real Transfer