CAMAR: Continuous Actions Multi-Agent Routing

2025-08-18

AI Summary

Existing multi-agent reinforcement learning (MARL) benchmarks lack standardized continuous-state-and-action-space challenges that jointly embody cooperation and competition in multi-agent path planning. To address this gap, we propose CAMAR—the first dedicated benchmark for multi-agent path planning in continuous domains, enabling joint modeling and evaluation of classical planners (e.g., RRT/RRT*) and MARL algorithms. Our method introduces a three-tiered evaluation protocol unifying assessment of coordination capability, planning efficiency, and policy interpretability; integrates a high-throughput simulation framework achieving up to 100,000 steps/second; and covers diverse, systematically designed scenarios. CAMAR fills a critical void in continuous-domain multi-agent planning benchmarks, significantly enhancing fairness and reproducibility in algorithmic comparison. Empirical results demonstrate its effectiveness in stress-testing both learning-based and sampling-based planners under realistic multi-agent dynamics.

Technology Category

• Multiagent Systems: Multiagent Planning
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Intelligent Robots: Motion and Path Planning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Multi-agent reinforcement learning (MARL) is a powerful paradigm for solving cooperative and competitive decision-making problems. While many MARL benchmarks have been proposed, few combine continuous state and action spaces with challenging coordination and planning tasks. We introduce CAMAR, a new MARL benchmark designed explicitly for multi-agent pathfinding in environments with continuous actions. CAMAR supports cooperative and competitive interactions between agents and runs efficiently at up to 100,000 environment steps per second. We also propose a three-tier evaluation protocol to better track algorithmic progress and enable deeper analysis of performance. In addition, CAMAR allows the integration of classical planning methods such as RRT and RRT* into MARL pipelines. We use them as standalone baselines and combine RRT* with popular MARL algorithms to create hybrid approaches. We provide a suite of test scenarios and benchmarking tools to ensure reproducibility and fair comparison. Experiments show that CAMAR presents a challenging and realistic testbed for the MARL community.

Problem

• CAMAR addresses multi-agent pathfinding with continuous actions.
• It integrates classical planning methods into MARL pipelines.
• CAMAR provides a challenging testbed for MARL algorithms.

Innovation

• Continuous actions multi-agent routing benchmark
• Integrates RRT and RRT* with MARL
• Three-tier evaluation protocol for tracking