EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revision

2026-08-07

AI Summary

This work addresses the limitations of existing automated multi-agent system design methods, which struggle to incorporate subsequent experience into reusable updates after initial optimization and often incur high token costs for high accuracy. The paper proposes EMAS, a framework that iteratively refines system topology and prompts through evidence-guided revision without updating large model parameters, by converting execution trajectories into structured diagnostic information. Its key innovation lies in a conditionally triggered revision mechanism based on repeated diagnostics and a validation-driven acceptance criterion, enabling stable, experience-driven evolution. Experiments demonstrate that EMAS achieves state-of-the-art or tied-best performance across four benchmarks and two large language models, improving task-weighted accuracy by up to 20.10% within two evolution rounds, boosting MBPP accuracy from 55.09% to 89.12%, and reducing token consumption by 62.2%.

Abstract

Many methods for automated multi-agent system design optimize prompts and topologies during an initial design stage and then deploy the resulting system unchanged on subsequent samples. Experience from these samples is rarely consolidated into reusable system updates, while accuracy-oriented designs may incur high token costs. We introduce EMAS (Evolving Multi-Agent System), which uses this experience to revise MAS topology and prompts without updating LLM parameters, either to improve accuracy or to reduce cost. EMAS converts traces into structured diagnoses that specify a revision operation and target. It generates a candidate revision only when the same diagnosis recurs across samples and applies it only if paired validation against the current MAS meets the corresponding acceptance criterion. Across four benchmarks and two LLMs, EMAS attains the highest task-weighted overall accuracy for both backbones and is best or tied in six of eight model--benchmark settings. Within two evolution epochs, EMAS achieves relative gains of 6.30% and 20.10% in task-weighted accuracy on Kimi-K2-6 and Qwen3.6-27B, respectively. On MBPP with Qwen3.6-27B, EMAS raises accuracy from 55.09% to 89.12% while reducing token use per task by 62.2%. These results show that EMAS can turn experience from new samples into reusable updates to MAS topology and prompts.

Problem

• multi-agent system
• system evolution
• experience consolidation
• prompt optimization
• token cost

Innovation

• Evidence-Guided Revision
• Multi-Agent System Evolution
• Prompt Optimization
• Topology Adaptation
• Token Efficiency