Environment-free Synthetic Data Generation for API-Calling Agents

2026-07-18

AI Summary

Training API-calling agents requires large-scale, high-quality trajectory data, yet conventional approaches rely on fully executable environments and pre-populated databases, limiting their scalability. This work proposes the first synthetic data generation framework that operates without access to a real execution environment, leveraging only API specifications. By orchestrating large language models to collaboratively generate tasks, simulate stateful API interactions, and filter trajectory quality, the method establishes an end-to-end synthetic pipeline. It achieves, for the first time, fully environment-free synthesis of API interaction trajectories. Evaluated on the AppWorld and OfficeBench benchmarks, models fine-tuned with this synthetic data demonstrate substantial performance gains, confirming the framework’s effectiveness and scalability.

Abstract

Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories. However, collecting such data at scale typically requires fully implemented environments with executable APIs and realistic, pre-populated backend databases, creating a major bottleneck for scalability. To overcome this, we propose an environment-free synthetic data generation approach that leverages LLMs as on-the-fly digital world models. Given only API specifications, our method generates trajectories mimicking interactions between an agent and a stateful environment. Specifically, an LLM first generates diverse tasks solvable with the provided APIs. A teacher agent then iteratively solves each task while an LLM simulator generates coherent synthetic API responses conditioned on the task context and simulation history. Finally, an LLM judge filters the trajectories to ensure the quality of the resulting dataset. We evaluate our approach on the challenging AppWorld and OfficeBench benchmarks, which include both information-retrieval and state-changing tasks. Fine-tuning models on our synthetic data yields significant performance gains, demonstrating that effective supervision for API-calling agents can be generated without any executable environment. Our results establish LLM-based API simulation as a practical, scalable solution for training agents across diverse API ecosystems.

Problem

• API-calling agents
• synthetic data generation
• environment-free
• trajectory data
• scalability bottleneck

Innovation

• environment-free
• synthetic data generation
• API-calling agents
• LLM-based simulation
• digital world model