When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

2026-07-17

AI Summary

This study addresses the unresolved question of when multi-agent systems (MAS) outperform single-agent systems (SAS). It formalizes MAS design as an information bottleneck optimization problem governed by a tunable parameter β, revealing that MAS achieve information compression through bounded relay messages within isolated local contexts, whereas SAS rely on shared contexts to accumulate full reasoning trajectories. Drawing on information bottleneck theory and supported by 18 controlled experiments across varying model scales and multiple benchmark tasks, the work demonstrates that MAS substantially enhance the performance of weaker models when communication relays approach sufficiency. However, for stronger models—capable of effectively leveraging redundant contextual information—the benefits of MAS diminish and can even become detrimental.

Abstract

LLM powered multi-agent systems (MAS) have emerged as a promising paradigm for complex tasks. However, their advantages over single-agent systems (SAS) remain unclear, with performance varying inconsistently across settings. Here, we provide an information bottleneck perspective on elucidating the differences between MAS and SAS. Specifically, our key observation is that a SAS accumulates its full reasoning trace in one shared context, while a MAS uses isolated local contexts connected by bounded relay messages. We show that, under infinite relay bandwidth, any SAS can be simulated by a MAS that transmits the full upstream context. Thus, the nontrivial advantage of MAS arises under bounded relays, where compression introduces a fundamental trade-off: reducing redundant context can improve efficiency, but may also incur loss of task-relevant information. We formalize this trade-off as an information bottleneck controlled by an effective parameter $β$, which captures how the balance shifts with model capability, and shows that MAS gains arise when context reduction outweighs relay information loss. We conduct 18 controlled experiments across five benchmarks and three model scales to validate our theoretical studies. We observe that MAS consistently helps when relays are near-sufficient, especially for weaker models. In contrast, MAS gains shrink or reverse when relays incur information loss, especially for stronger models that can already extract useful information from redundant context and thus gain little from compression. Our study shows that multi-agent design is fundamentally an information-bottleneck optimization problem. This perspective explains when bounded inter-agent communication helps or hurts.

Problem

• multi-agent systems
• information bottleneck
• bounded communication
• context compression
• performance advantage

Innovation

• information bottleneck
• multi-agent systems
• context compression
• relay communication
• reasoning efficiency