Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM

2026-07-29

AI Summary

Current evaluation methods struggle to disentangle whether the effectiveness of latent communication in multi-agent large language models stems from the mere presence of messages, the specific content of examples, or information about other agents. This work proposes a causal auditing framework that introduces, for the first time, a controlled message replacement protocol to systematically substitute messages at the sender–receiver boundary, thereby decomposing and quantifying the independent contributions of example-specific content and other-agent information. Experiments with Qwen3-4B and Qwen3-8B on GSM8K, ARC-C, and MATH-500 reveal that model scale significantly influences how information is utilized and that the contributions of these two information types are separable, exposing the limitations of conventional accuracy-based metrics.

Abstract

Latent communication in large language model (LLM)-based multi-agent systems (MAS) transmits continuous internal representations instead of text, but greater representational capacity does not establish that the receiver uses task-relevant information. End-task performance alone also cannot reveal whether an observed effect depends on message presence, content generated for the evaluated example, or information supplied by a separate agent. We introduce a causal audit that applies controlled message replacements at the boundary where the sender-produced representation enters the receiver. Four message settings support five measurements of encoded sender information, receiver sensitivity to message presence and identity, the task value of example-specific content, and the additional value supplied by a separate agent. We apply the audit to latent relay with Qwen3-4B and Qwen3-8B on GSM8K, ARC-C, and MATH-500. On GSM8K, the Qwen3-4B overall performance effect of -1.00 percentage point decomposes into a -6.17-point effect retained by an other-example message and a +5.17-point effect attributable to example-specific content; both component directions reverse at 8B. On MATH-500, the Qwen3-4B gain of 15.00 points comprises 8.33 points retained by an other-example message and 6.67 points attributable to example-specific content, while the 8B gain is dominated by the former component. Self-substitution comparisons further show that example-specific content and other-agent value are distinct. These results show that aggregate accuracy does not identify how a latent message affects the receiver and motivate controlled message comparisons as a standard evaluation for latent communication.

Problem

• latent communication
• multi-agent systems
• causal audit
• large language models
• message content

Innovation

• causal audit
• latent communication
• multi-agent LLM
• controlled message replacement
• information decomposition