Can AI Models Direct Each Other? Organizational Structure as a Probe into Training Limitations

2026-03-27

AI Summary

This study investigates whether expensive AI models can effectively guide cheaper counterparts in software engineering tasks, revealing limitations in current multi-agent systems stemming from monolithic training that lacks delegation, scope control, and mode-switching capabilities. To address this, the authors propose a Manager-Worker dual-agent framework: a text-only “Manager” handles problem analysis, task delegation, and code review, while an execution-capable “Worker” performs concrete modifications, with an explicit organizational structure enhancing collaboration efficiency. Experiments on SWE-bench Lite show that a strong Manager guiding a weak Worker achieves a 62% success rate—comparable to a strong single-agent system’s 60%—while substantially reducing high-cost token usage. Conversely, a weak Manager degrades performance, confirming that effective guidance relies on genuine capability disparity. Moreover, active guidance outperforms passive review by 11 percentage points, demonstrating that thoughtful architectural design can compensate for insufficient training.

Technology Category

• Multiagent Systems: Teamwork
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Can an expensive AI model effectively direct a cheap one to solve software engineering tasks? We study this question by introducing ManagerWorker, a two-agent pipeline where an expensive "manager" model (text-only, no code execution) analyzes issues, dispatches exploration tasks, and reviews implementations, while a cheap "worker" model (with full repo access) executes code changes. We evaluate on 200 instances from SWE-bench Lite across five configurations that vary the manager-worker relationship, pipeline complexity, and model pairing. Our findings reveal both the promise and the limits of multi-agent direction: (1) a strong manager directing a weak worker (62%) matches a strong single agent (60%) at a fraction of the strong-model token usage, showing that expensive reasoning can substitute for expensive execution; (2) a weak manager directing a weak worker (42%) performs worse than the weak agent alone (44%), demonstrating that the directing relationship requires a genuine capability gap--structure without substance is pure overhead; (3) the manager's value lies in directing, not merely reviewing--a minimal review-only loop adds just 2pp over the baseline, while structured exploration and planning add 11pp, showing that active direction is what makes the capability gap productive; and (4) these behaviors trace to a single root cause: current models are trained as monolithic agents, and splitting them into director/worker roles fights their training distribution. The pipeline succeeds by designing around this mismatch--keeping each model close to its trained mode (text generation for the manager, tool use for the worker) and externalizing organizational structure to code. This diagnosis points to concrete training gaps: delegation, scoped execution, and mode switching are skills absent from current training data.

Problem

• multi-agent coordination
• AI delegation
• training limitations
• organizational structure
• capability gap

Innovation

• multi-agent collaboration
• organizational structure
• model delegation
• training distribution mismatch
• software engineering automation