Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments

2025-01-18

AI Summary

To address the limited adaptability and learning capability of LLM-based agents in real-world environments—such as codebases, web interfaces, and desktop applications—due to scarce high-quality interactive data, this paper proposes a human-annotation-free, data-driven adaptive framework. Its core innovation is the novel “reverse construction” mechanism: automatically synthesizing interactive trajectories from documentation and distilling task-specific instructions therefrom. We further design an agent-optimized retrieval method that jointly supports task-aware retrieval augmentation, abstraction of interaction history summaries, and synergistic enhancement of both training-free in-context learning (ICL) and training-based fine-tuning. Evaluated on four major benchmarks—including SWE-bench—the framework achieves +12.2% ICL performance gain (Claude-3.5) and +19.5% fine-tuning improvement (Codestral-22B), with reverse construction alone contributing up to +14.0% absolute gain.

Technology Category

Natural Language Processing: Code Generation / Program Synthesis from Natural LanguageMachine Learning: Large Multimodal Models (LMMs)Search and Optimization: Learning to Search

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Autonomous agents powered by large language models (LLMs) have the potential to enhance human capabilities, assisting with digital tasks from sending emails to performing data analysis. The abilities of existing LLMs at such tasks are often hindered by the lack of high-quality agent data from the corresponding environments they interact with. We propose Learn-by-interact, a data-centric framework to adapt LLM agents to any given environments without human annotations. Learn-by-interact synthesizes trajectories of agent-environment interactions based on documentations, and constructs instructions by summarizing or abstracting the interaction histories, a process called backward construction. We assess the quality of our synthetic data by using them in both training-based scenarios and training-free in-context learning (ICL), where we craft innovative retrieval approaches optimized for agents. Extensive experiments on SWE-bench, WebArena, OSWorld and Spider2-V spanning across realistic coding, web, and desktop environments show the effectiveness of Learn-by-interact in various downstream agentic tasks -- baseline results are improved by up to 12.2% for ICL with Claude-3.5 and 19.5% for training with Codestral-22B. We further demonstrate the critical role of backward construction, which provides up to 14.0% improvement for training. Our ablation studies demonstrate the efficiency provided by our synthesized data in ICL and the superiority of our retrieval pipeline over alternative approaches like conventional retrieval-augmented generation (RAG). We expect that Learn-by-interact will serve as a foundation for agent data synthesis as LLMs are increasingly deployed at real-world environments.

Problem

• Large Language Models
• Real-world Adaptation
• Interaction Quality

Innovation

• Learn-by-interact framework
• Autonomous learning
• Efficiency enhancement