WorkflowOps: Learning Agent Collaboration Priors for Multi-Agent Workflow Orchestration

2026-10-06

AI Summary

This study addresses the absence of memory mechanisms in existing multi-agent orchestration layers, which prevents the reuse of historical experience for novel tasks. We propose an orchestration framework that learns collaborative priors by extracting patterns from historical workflows via a collaboration transition probability matrix to guide DAG construction. Furthermore, we design a semantic matching-based mechanism for dynamic agent expansion and injection. To reduce computational overhead, a hierarchical strategy combining sentence embedding pre-screening with LLM verification is employed, alongside graph optimization to maximize parallelism. Experiments on mixed benchmarks spanning code, mathematics, and question answering demonstrate that the proposed framework significantly improves end-to-end pass rates and reduces LLM routing calls by over 80%, yielding particularly pronounced gains on structured tasks.

Abstract

Multi-agent systems are increasingly deployed for complex knowledge work, yet their orchestration layers remain largely memoryless: each new task is decomposed, assigned, and executed from scratch with no benefit from prior successful executions. We present WorkflowOps, a multi-agent workflow orchestration framework that learns agent collaboration priors from historical workflows and expands its agent pool on demand to cover new capability requirements. Our approach introduces three coupled mechanisms. First, a transition probability matrix captures pairwise agent collaboration frequencies from past workflows and applies them as soft guidance during DAG workflow construction through intra-layer ordering optimization, probability-thresholded edge suggestion, and transitive reduction for parallelism maximization. Second, a sufficiency-driven agent creation loop detects capability gaps via semantic matching scores, generates specialized agents through an LLM, and simultaneously injects them into the collaboration matrix, so that newly created agents are immediately usable with predicted collaboration priors. Third, a layered semantic matching strategy uses pre-trained sentence embeddings for fast, deterministic capability matching as a first pass, invoking LLM verification only for low-confidence cases, thereby reducing LLM routing calls by over 80\% compared to pure-LLM approaches. Experiments on mixed code, math, and question-answering suites show that WorkflowOps improves end-to-end pass rates over recent workflow-construction baselines, with the largest gains on structured, decomposable tasks where past agent handoff patterns transfer.

Problem

• Multi-agent systems
• Workflow orchestration
• Agent collaboration priors
• Capability matching
• Memoryless execution

Innovation

• Multi-Agent Workflow Orchestration
• Collaboration Priors
• Transition Probability Matrix
• Dynamic Agent Creation
• Layered Semantic Matching