CoMemBench: Benchmarking Collaborative Memory Boundaries across Multi-Agent Workflow Topologies

2026-09-26

AI Summary

This study addresses the challenge of quantitatively evaluating memory sharing and isolation boundaries in multi-agent workflows by introducing, for the first time, a formal definition of collaborative memory boundaries. Methodologically, it constructs a composite workflow benchmark based on source dependency graphs, integrating node specifications, verifiable artifact handoffs, and native evaluators to systematically assess the robustness of information sharing and isolation across diverse topologies. The findings reveal a significant trade-off between context breadth and isolation capability, with system performance rankings varying substantially across different topological structures. By bridging the gap in topology-conditioned memory evaluation, this work provides critical empirical evidence to inform architectural design decisions for multi-agent collaborative systems.

Abstract

Multi-agent workflows require task-relevant information to be shared across agents, while irrelevant, stale, unverified, or incompatible information must remain isolated. We call this task-conditioned scope of information a collaborative memory boundary. Workflow topology determines which intermediate artifacts are applicable to which downstream workers and when they cease to be valid, thereby providing a structural stress dimension for sharing and isolation. Existing memory benchmarks primarily evaluate retention and retrieval, whereas multi-agent benchmarks emphasize coordination and end-to-end completion, leaving topology-conditioned memory boundaries largely unmeasured. We introduce CoMemBench, an execution-grounded benchmark for collaborative memory sharing and isolation across multi-agent workflow topologies. It constructs 800 composite workflows across four domains from source-grounded dependency graphs, with node-local specifications, verifiable artifact handoffs, native evaluators, and matched isolation challenges. CoMemBench measures workflow completion, verified node progress, required-handoff reliability, isolation robustness, and token cost. Experiments reveal a sharing-isolation trade-off: broader context improves information availability but can weaken isolation, while system rankings shift across topologies and artifact violations.

Problem

• multi-agent workflows
• collaborative memory boundary
• workflow topology
• benchmarking
• information isolation

Innovation

• Collaborative Memory Boundary
• Multi-Agent Workflow Topologies
• Benchmark
• Information Isolation
• Artifact Handoffs