FlowForge: Guiding the Creation of Multi-agent Workflows with Design Space Visualization as a Thinking Scaffold

2025-07-21

AI Summary

Multi-agent workflow design suffers from an expansive design space, heavy reliance on expert intuition, and a tendency to converge prematurely to suboptimal solutions or engage in inefficient trial-and-error. To address these challenges, this paper proposes a three-layer progressive design-space modeling framework that decouples task planning, agent orchestration, and optimization—integrated with a design pattern library and context-aware recommendation algorithms—to realize an interactive, visual tool supporting in-situ guidance and multi-criteria comparison. Our contributions are threefold: (1) the first systematic, structured formalization of the multi-agent design space; (2) visualization-enabled cognitive scaffolding that closes the “perceive–reason–decide” loop; and (3) context-driven real-time recommendations that enhance both design quality and efficiency. User studies and case validations demonstrate significant reductions in design cycle time, improved systematicity in solution exploration, and higher recommendation adoption rates.

Technology Category

• Multiagent Systems: Mechanism Design
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-agent workflows have become an effective strategy for tackling complicated tasks by decomposing them into multiple sub-tasks and assigning them to specialized agents. However, designing optimal workflows remains challenging due to the vast and intricate design space. Current practices rely heavily on the intuition and expertise of practitioners, often resulting in design fixation or an unstructured, time-consuming exploration of trial-and-error. To address these challenges, this work introduces FLOWFORGE, an interactive visualization tool to facilitate the creation of multi-agent workflow through i) a structured visual exploration of the design space and ii) in-situ guidance informed by established design patterns. Based on formative studies and literature review, FLOWFORGE organizes the workflow design process into three hierarchical levels (i.e., task planning, agent assignment, and agent optimization), ranging from abstract to concrete. This structured visual exploration enables users to seamlessly move from high-level planning to detailed design decisions and implementations, while comparing alternative solutions across multiple performance metrics. Additionally, drawing from established workflow design patterns, FLOWFORGE provides context-aware, in-situ suggestions at each level as users navigate the design space, enhancing the workflow creation process with practical guidance. Use cases and user studies demonstrate the usability and effectiveness of FLOWFORGE, while also yielding valuable insights into how practitioners explore design spaces and leverage guidance during workflow development.

Problem

• Challenges in designing optimal multi-agent workflows
• Overcoming design fixation and unstructured trial-and-error
• Lack of guided tools for workflow design exploration

Innovation

• Interactive visualization tool for multi-agent workflows
• Structured visual exploration of design space
• In-situ guidance with design patterns