Hierarchical Reinforcement Learning for Cooperative Air-Ground Delivery in Urban System

2026-02-13

Technology Category

• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Cooperative air-ground delivery has emerged as a promising logistics paradigm by leveraging the complementary strengths of UAVs and ground carriers. However, effective dispatching in such heterogeneous systems faces two critical challenges: i) the heterogeneity between flight and road dynamics, ii) the scalability bottleneck raised by the exponential decision variables in large-scale fleets. To address these challenges, we propose HRL4AG, a Hierarchical Reinforcement Learning framework for cooperative Air-Ground delivery. Specifically, HRL4AG employs a high-level manager to tackle the scalability bottleneck by decomposing the joint action space, and mode-specific workers that encode distinct flight and road dynamics to address the heterogeneity. Furthermore, a novel internal reward mechanism is designed to guide the hierarchical policy learning, addressing the credit assignment problem in sparse-reward settings. Extensive experiments on two real-world datasets and an evaluation platform demonstrate that HRL4AG significantly outperforms state-of-the-art baselines, improving the delivery success rate by up to 26% while achieving an 80-fold increase in computational efficiency.

Problem

• cooperative air-ground delivery
• heterogeneity
• scalability bottleneck
• urban logistics
• multi-agent dispatching

Innovation

• Hierarchical Reinforcement Learning
• Air-Ground Collaboration
• Heterogeneous Multi-Agent Systems
• Scalable Dispatching
• Credit Assignment