MOSAIC: Modular Orchestration for Structured Agentic Intelligence and Composition

2026-05-30

AI Summary

This work addresses the lack of structured, verifiable, and reusable decision mechanisms in existing automated machine learning approaches for model selection. It proposes a semantic task profiling–based structured agent framework that leverages retrieval-augmented generation of historical cases and code modules to construct an intermediate representation blueprint encompassing modeling components, composition logic, and execution constraints. By integrating code execution feedback with a failure-aware reinforcement learning strategy, the framework enables memory-driven, traceable, multi-stage search optimization. Evaluated on financial time-series forecasting and generation tasks, the method significantly outperforms both conventional AutoML systems and current agent-based baselines, achieving consistent improvements in task performance, execution success rate, and decision interpretability.

Abstract

Automated data science is a structured model-selection problem. A solution must choose data transformations, feature representations, architecture, training procedure, evaluation protocol, and refinement strategy for a task. AutoML systems automate parts of this process, but typically search within predefined pipeline, model, and hyperparameter spaces. LLM-based agents offer greater flexibility through retrieval, code generation, and execution feedback, yet their modelling decisions are often unstructured, difficult to verify, and hard to reuse. We introduce \textsc{MOSAIC} (Modular Orchestration for Structured Agentic Intelligence and Composition), a structured agentic framework for memory-grounded model selection and workflow construction. Given a task and dataset, \textsc{MOSAIC} builds a semantic task profile, retrieves prior cases and source-code modules, and constructs a blueprint: an intermediate representation specifying selected modelling components, composition, interface constraints, and execution requirements. This blueprint turns model selection into a staged, context-grounded search and grounds LLM-based code generation in retrieved evidence rather than unconstrained synthesis. Candidate models are validated by execution and refined using diagnostic feedback, training traces, task metrics, and a failure-aware reinforcement learning policy. We instantiate \textsc{MOSAIC} on financial time-series forecasting and generation, where models must satisfy predictive accuracy, distributional fidelity, execution reliability, and downstream financial criteria such as risk and tail behaviour. Experiments against AutoML and agentic baselines show that \textsc{MOSAIC} improves task performance, execution success, and decision traceability, demonstrating the value of treating automated data science as structured, reusable, and execution-grounded model selection.

Problem

• Automated Data Science
• Model Selection
• Structured Agentic Intelligence
• Workflow Composition
• LLM-based Agents

Innovation

• structured agentic framework
• blueprint-based model selection
• memory-grounded orchestration
• execution-grounded LLM code generation
• failure-aware reinforcement learning