🤖 AI Summary
This work addresses the challenge of degraded information timeliness and action misalignment in multi-agent collaboration caused by cross-timestep communication delays. It formalizes delayed communication as a Decentralized Communication Partially Observable Markov Game (DeComm-POMG) and introduces a Communication Gain and Delay Cost (CGDC) decomposition mechanism. Building upon this, the authors propose an adaptive communication strategy that triggers message exchange only when the net benefit is positive. They develop CDCMA, an actor-critic framework integrating future observation prediction, CGDC-guided attention, and dynamic message requesting to enhance information fusion efficiency. Experimental results demonstrate that the proposed method significantly outperforms baseline approaches across diverse delay settings in Cooperative Navigation, Predator-Prey, and SMAC tasks, achieving superior performance, robustness, and generalization capability.
📝 Abstract
Communication is essential for coordination in \emph{cooperative} multi-agent reinforcement learning under partial observability, yet \emph{cross-timestep} delays cause messages to arrive multiple timesteps after generation, inducing temporal misalignment and making information stale when consumed.
We formalize this setting as a delayed-communication partially observable Markov game (DeComm-POMG) and decompose a message's effect into \emph{communication gain} and \emph{delay cost}, yielding the Communication Gain and Delay Cost (CGDC) metric.
We further establish a value-loss bound showing that the degradation induced by delayed messages is upper-bounded by a discounted accumulation of an information gap between the action distributions induced by timely versus delayed messages.
Guided by CGDC, we propose \textbf{CDCMA}, an actor--critic framework that requests messages only when predicted CGDC is positive, predicts future observations to reduce misalignment at consumption, and fuses delayed messages via CGDC-guided attention.
Experiments on no-teammate-vision variants of Cooperative Navigation and Predator Prey, and on SMAC maps across multiple delay levels show consistent improvements in performance, robustness, and generalization, with ablations validating each component.