Heterogeneous Multi-Expert Reinforcement Learning for Long-Horizon Multi-Goal Tasks in Autonomous Forklifts

2026-01-12

AI Summary

This work addresses the challenge faced by autonomous forklifts in unstructured warehouse environments, where long-horizon multitask execution requires balancing large-scale efficient navigation with high-precision object manipulation. To this end, the authors propose a Heterogeneous Multi-Expert Reinforcement Learning framework (HMER), which employs a semantic task planner to decompose complex tasks into specialized sub-policies—each optimized for either macroscopic navigation or microscopic manipulation. By integrating imitation learning with reinforcement learning, the framework effectively mitigates the sparse reward problem. Evaluated in Gazebo simulations, the approach achieves a 94.2% task success rate—substantially outperforming the baseline of 62.5%—while reducing operation time by 21.4% and maintaining placement errors below 1.5 cm, thereby significantly enhancing overall system performance.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Autonomous mobile manipulation in unstructured warehouses requires a balance between efficient large-scale navigation and high-precision object interaction. Traditional end-to-end learning approaches often struggle to handle the conflicting demands of these distinct phases. Navigation relies on robust decision-making over large spaces, while manipulation needs high sensitivity to fine local details. Forcing a single network to learn these different objectives simultaneously often causes optimization interference, where improving one task degrades the other. To address these limitations, we propose a Heterogeneous Multi-Expert Reinforcement Learning (HMER) framework tailored for autonomous forklifts. HMER decomposes long-horizon tasks into specialized sub-policies controlled by a Semantic Task Planner. This structure separates macro-level navigation from micro-level manipulation, allowing each expert to focus on its specific action space without interference. The planner coordinates the sequential execution of these experts, bridging the gap between task planning and continuous control. Furthermore, to solve the problem of sparse exploration, we introduce a Hybrid Imitation-Reinforcement Training Strategy. This method uses expert demonstrations to initialize the policy and Reinforcement Learning for fine-tuning. Experiments in Gazebo simulations show that HMER significantly outperforms sequential and end-to-end baselines. Our method achieves a task success rate of 94.2\% (compared to 62.5\% for baselines), reduces operation time by 21.4\%, and maintains placement error within 1.5 cm, validating its efficacy for precise material handling.

Problem

• Autonomous forklifts
• Long-horizon multi-goal tasks
• Navigation-manipulation conflict
• Heterogeneous reinforcement learning
• Sparse exploration

Innovation

• Heterogeneous Multi-Expert Reinforcement Learning
• Semantic Task Planner
• Hybrid Imitation-Reinforcement Training
• Long-Horizon Multi-Goal Tasks
• Autonomous Forklifts