Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details

2026-08-04

AI Summary

Current evaluations of zero-shot coordination (ZSC) algorithms predominantly rely on varying random seeds within a single implementation, which fails to capture performance discrepancies arising from independently developed implementations and thus underestimates algorithmic robustness. This work proposes a cross-implementation cross-play evaluation paradigm that systematically assesses ZSC algorithms under diverse implementation details—such as network architectures and optimizers—to provide a more realistic measure of their generalization capability. Experiments based on the Other-Play algorithm demonstrate consistent performance between standard cross-seed evaluation and the stricter cross-implementation setting, confirming the algorithm’s robustness while underscoring the complementary value of the proposed evaluation framework for advancing ZSC research.

Abstract

AI agents deployed in real-world settings must be capable of coordinating with humans and other AI agents they have not encountered before. Zero-shot coordination (ZSC) algorithms aim to achieve this by specifying high-level learning rules such that independently engineered agents can coordinate with each other at test time. Rigorous evaluation of ZSC algorithms remains difficult: ideally, multiple independent implementations of each proposed algorithm must be used, reflecting the variation that arises when independent parties interpret and implement the same specification. In practice, however, ZSC algorithms have almost exclusively been evaluated using a single implementation trained across different random seeds, with only a handful of works additionally varying the neural network architecture. This leaves open questions about robustness to specification ambiguities and implementation details. In this work, we provide the first systematic evaluation of this robustness. We introduce a new evaluation scheme, cross-implementation cross-play, varying implementation details that prior work has shown to affect the performance of multi-agent reinforcement learning (MARL) algorithms, and we evaluate Other-Play, a popular ZSC algorithm, with this scheme. Our findings are encouraging and suggest that, for Other-Play, the standard ZSC evaluation is, in fact, a reasonable proxy for this more thorough cross-implementation evaluation.

Problem

• Zero-Shot Coordination
• Robustness
• Implementation Details
• Cross-Play Evaluation
• Multi-Agent Reinforcement Learning

Innovation

• zero-shot coordination
• cross-implementation evaluation
• multi-agent reinforcement learning
• robustness
• Other-Play