A Flexible Multi-Agent LLM-Human Framework for Fast Human Validated Tool Building

2025-12-01

AI Summary

To address the challenges of poor human-intent alignment, slow domain adaptation, and high human-feedback costs in tool construction, this paper proposes an expert-in-the-loop multi-agent LLM collaboration framework. It introduces four specialized agents—Coach, Coder, Critic, and Reuser—alongside a dynamic reinforcement prompting mechanism to enable system-level in-context learning, real-time intent calibration, and reusable tool accumulation. By integrating multi-agent collaborative reasoning, human feedback reinforcement learning (HFRL), and dynamic prompt engineering, the framework significantly reduces domain adaptation time (by 62% on average) and human feedback iterations (by 57%) for complex tasks such as scientific document generation. Its core innovation lies in deeply embedding expert knowledge into a closed-loop multi-agent collaboration, thereby enabling efficient, scalable, and intent-driven tool evolution.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We introduce CollabToolBuilder, a flexible multiagent LLM framework with expert-in-the-loop (HITL) guidance that iteratively learns to create tools for a target goal, aligning with human intent and process, while minimizing time for task/domain adaptation effort and human feedback capture. The architecture generates and validates tools via four specialized agents (Coach, Coder, Critic, Capitalizer) using a reinforced dynamic prompt and systematic human feedback integration to reinforce each agent's role toward goals and constraints. This work is best viewed as a system-level integration and methodology combining multi-agent in-context learning, HITL controls, and reusable tool capitalization for complex iterative problems such as scientific document generation. We illustrate it with preliminary experiments (e.g., generating state-of-the-art research papers or patents given an abstract) and discuss its applicability to other iterative problem-solving.

Problem

• Develops a multi-agent LLM framework for tool creation
• Integrates human feedback to align tools with user intent
• Accelerates adaptation for complex iterative tasks like document generation

Innovation

• Multi-agent LLM framework with human-in-the-loop guidance
• Four specialized agents generate and validate tools iteratively
• Reinforced dynamic prompts and systematic human feedback integration