Technology Category
Intelligent Robots: Localization, Mapping, and NavigationMachine Learning: Reinforcement LearningHumans and AI: Human-Aware Planning and Behavior Prediction
Hierarchical Reinforcement Learning in Multi-Goal Spatial Navigation with Autonomous Mobile Robots
🤖 AI Summary
To address the low sample efficiency and poor task success rate of conventional reinforcement learning (RL) in sparse-reward multi-target spatial navigation for autonomous mobile robots, this paper proposes a hierarchical RL (HRL) framework featuring automated subgoal generation and dynamic termination detection. The framework decouples navigation into high-level subgoal planning and low-level action execution, jointly optimizing subgoal abstraction and a learnable termination function. Evaluated in a Gazebo+ROS simulation environment, our method achieves a 42% improvement in sample efficiency and a 31% increase in task success rate over end-to-end PPO, while demonstrating enhanced robustness to reward sparsity. Moreover, we provide the first systematic analysis of how subgoal generation strategy (manual vs. automatic) and termination frequency affect navigation performance—empirically validating the effectiveness and generalization advantage of automatic-subgoal-driven HRL in complex navigation tasks.
Technology Category
Application Category
📝 Abstract
Hierarchical reinforcement learning (HRL) is hypothesized to be able to take advantage of the inherent hierarchy in robot learning tasks with sparse reward schemes, in contrast to more traditional reinforcement learning algorithms. In this research, hierarchical reinforcement learning is evaluated and contrasted with standard reinforcement learning in complex navigation tasks. We evaluate unique characteristics of HRL, including their ability to create sub-goals and the termination function. We constructed experiments to test the differences between PPO and HRL, different ways of creating sub-goals, manual vs automatic sub-goal creation, and the effects of the frequency of termination on performance. These experiments highlight the advantages of HRL and how it achieves these advantages.
Problem
Research questions and friction points this paper is trying to address.
Evaluating HRL vs standard RL in complex navigation tasks
Assessing sub-goal creation methods in hierarchical reinforcement learning
Analyzing termination frequency impact on HRL performance
Innovation
Methods, ideas, or system contributions that make the work stand out.
Hierarchical reinforcement learning for sparse rewards
Automatic sub-goal creation in navigation tasks
Termination function optimization in HRL
🔎 Similar Papers
No similar papers found.
