Bel Esprit: Multi-Agent Framework for Building AI Model Pipelines

2024-12-19

arXiv.org

AI Summary

Complex AI tasks often involve ambiguous user requirements that single-model systems struggle to address effectively. Method: This paper proposes an automated AI pipeline generation method based on multi-agent collaboration. It introduces a novel conversational multi-agent framework featuring a requirement clarification dialogue engine that progressively parses natural-language queries and resolves ambiguities. Integrated with a model-adaptive retrieval mechanism and a pipeline validation module, the system enables end-to-end generation of executable and evaluable AI workflows. A synthetic-data-driven evaluation strategy ensures output quality. Contribution/Results: Experiments demonstrate that the system significantly improves pipeline construction accuracy under ambiguous queries, outperforming baseline approaches. The framework is open-sourced, and a free-to-use platform is publicly available.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

As the demand for artificial intelligence (AI) grows to address complex real-world tasks, single models are often insufficient, requiring the integration of multiple models into pipelines. This paper introduces Bel Esprit, a conversational agent designed to construct AI model pipelines based on user-defined requirements. Bel Esprit employs a multi-agent framework where subagents collaborate to clarify requirements, build, validate, and populate pipelines with appropriate models. We demonstrate the effectiveness of this framework in generating pipelines from ambiguous user queries, using both human-curated and synthetic data. A detailed error analysis highlights ongoing challenges in pipeline construction. Bel Esprit is available for a free trial at https://belesprit.aixplain.com.

Problem

• Integrating multiple AI models into pipelines for complex tasks
• Clarifying user requirements to construct appropriate model pipelines
• Addressing challenges in pipeline construction from ambiguous queries

Innovation

• Multi-agent framework for AI pipelines
• Conversational agent clarifies user requirements
• Generates pipelines from ambiguous queries