🤖 AI Summary
This work addresses the vulnerability of decentralized multi-agent path planning to minor adversarial perturbations in observations, which can lead to erratic behaviors and system-wide congestion. To enhance resilience, the study introduces certified robustness into this domain for the first time, proposing two training strategies: adversarial training based on Adv-PPO and a fine-tuning approach incorporating the MACER stochastic smoothing regularizer. Both methods significantly improve robustness without altering the underlying network architecture or deployment pipeline. Experimental results on an 8×8 grid with four agents demonstrate that worst-case success rates increase from 2.5% to 59.2% with Adv-PPO alone and further to 77.5% ± 6.0% when combined with MACER, while incurring less than 1% performance degradation under unperturbed (clean) conditions.
📝 Abstract
Decentralized multi-agent path finding (MAPF) routes a team of agents on a shared grid, each acting from its own local view. The standard solution trains one shared neural policy with Proximal Policy Optimization (PPO), a popular on-policy reinforcement learning algorithm. Such a policy works well on clean observations, but a small input perturbation on one agent often changes its action, which then blocks a neighbour, and the team jams. In this paper we present two training recipes that keep the same network and the same deployment loop, yet make the policy hold up under perturbed observations. The first recipe, Adv-PPO, trains the shared policy against worst-case perturbations of its own input and selects the checkpoint by performance under adversarial perturbation. The second recipe, Adv-PPO+MACER, fine-tunes that checkpoint with a small on-policy smoothness term whose gradient follows the certified radius of randomized smoothing. On POGEMA with 8x8 maps and four agents, the unprotected PPO policy reaches 95.8% clean success but only 2.5% under the strongest attack. Adv-PPO recovers worst-case success to 59.2% at one percentage point of clean cost. Adv-PPO+MACER recovers it to 77.5% +/- 6.0% across three independent seeds at less than one percentage point of clean cost. We support these numbers with per-attack curves, a certified action-stability sanity check (which measures the smoothed-policy wrapper, not the deployed argmax policy), and side-by-side rollout storyboards that show the failure mode and the fix inside one environment instance.