PRIMAL3: Pathfinding via Reinforcement and Imitation Multi-Agent Learning - Leveraging LaCAM3

2026-08-05

AI Summary

This work addresses the coordination challenges of ultra-large-scale multi-agent path finding (MAPF) in topologically critical regions—such as bottlenecks and dead ends—by proposing a scalable approach that integrates reinforcement learning with imitation learning. The method is trained via LaCAM3-guided label-smoothed imitation learning and incorporates a PIBT module to refine actions, enabling efficient collision-free navigation without relying on LaCAM3 during inference. Key innovations include a topology-aware communication mechanism, a LaCAM3 intervention strategy based on policy entropy, and a hybrid framework that combines structured prioritization with learned action refinement. This approach achieves, for the first time, city-scale MAPF solutions involving up to one hundred thousand agents, substantially outperforming existing learning-based methods and demonstrating practical deployability in real robotic systems.

Abstract

We present PRIMAL3, an ultra-large-scale learning-based framework for multi-agent pathfinding (MAPF) that integrates reinforcement learning, topology-aware communication, LaCAM3-guided training, and PIBT-based action refinement. PRIMAL3 targets failures at topologically critical states, where agents must coordinate decisively around bottlenecks, dead ends, and persistent conflicts. Each agent is represented using features derived from cut vertices, dead-end regions, shortest-path distances, and blocking estimates. Two complementary graphs capture agent interactions: a same-direction following graph propagates multihop context along compatible paths, while a different-direction conflict graph differentiates agents competing for shared space through masked attention and relative features. During training, we propose to let policy entropy identify uncertain agents, for which LaCAM3 provides confidence-triggered action interventions and label-smoothed imitation targets. During execution, a priority-aware PIBT module refines the proposed joint actions using persistent, learned, and distance-aware priorities together with policy-aware fallback preferences while maintaining collision-free execution. The resulting framework combines learned exploration with structured expert guidance without requiring LaCAM3 at inference. Experiments demonstrate that PRIMAL3 substantially outperforms state-of-the-art learning-based baselines and scales to ultra-large instances with up to city-level 100,000 agents. Real-world experiments further demonstrate the feasibility of deploying PRIMAL3 on physical robotic systems and ablation studies validate the individual contributions the components we proposed. Project page: https://marmotlab.github.io/PRIMAL3/

Problem

• multi-agent pathfinding
• topological critical states
• bottlenecks
• dead ends
• persistent conflicts

Innovation

• multi-agent pathfinding
• topology-aware communication
• LaCAM3-guided training
• PIBT-based action refinement
• reinforcement and imitation learning