DEALS: Decentralized Expertise-Aware Load Serving for Multi-Agent LLM Systems

2026-09-27

AI Summary

This study addresses the poor scalability of centralized control and the high routing overhead of decentralized approaches in multi-agent LLM systems by proposing the DEALS framework. DEALS adopts a decentralized architecture that enables task self-organization through local queues, concurrent executors, and a dynamic routing algorithm based on backlog and success rate, eliminating the need to train dedicated routers or invoke LLMs for decision-making. Experimental results demonstrate that DEALS significantly improves answer accuracy and system throughput across both homogeneous and heterogeneous agent pools. By effectively balancing agent expertise with workload distribution, the proposed method achieves low-cost decentralized coordination and cross-agent task recovery.

Abstract

Multi-agent systems (MAS) have recently emerged as an effective approach for coordinating large language model (LLM)-based agents to solve complex tasks through structured interactions. In practice, MASs often handle a stream of heterogeneous and complex tasks, requiring agents to decompose each task and then self-organize and self-evolve to adapt to incoming tasks while sharing execution resources. However, most early approaches to MASs rely on centralized controllers or fixed coordination patterns, which can limit scalability or adaptability. In contrast, existing decentralized and dynamic MASs often require training dedicated routers or invoking LLMs for agent selection, resulting in substantial computational costs and coordination overhead. To address these challenges and enable efficient task-level self-organization and self-evolution for task- and workload-level collaboration, we propose Decentralized Expertise-Aware Load Serving (DEALS), a decentralized and low-complexity framework that enables agents to self-organize and dynamically route concurrent tasks for processing. Specifically, each agent maintains local queues of incoming tasks, and its router decides whether to process a task locally or forward it to a neighbor based on differences in backlog and success rate. Meanwhile, executors process independent tasks concurrently within and across agents, and partially solved tasks can be resumed by other agents. Experiments show that DEALS not only improves performance along multiple dimensions (e.g., answer accuracy and task throughput) in both homogeneous and heterogeneous agent pools, but also balances agent expertise and workload in a self-organized manner, enabling effective decentralized coordination.

Problem

• Multi-Agent Systems
• Large Language Models
• Decentralized Coordination
• Task Routing
• Load Balancing

Innovation

• Decentralized Multi-Agent Systems
• Dynamic Task Routing
• Self-Organization
• Load Balancing
• Large Language Models