SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement

2025-04-04

AI Summary

LLMs exhibit poor generalization in novel environments and non-standard action spaces, struggling with autonomous exploration and action understanding. To address this, we propose a virtual scene synthesis framework featuring the first learnable, symbolic scene generation mechanism operating directly within the action space. This transforms environmental exploration into a controllable and evaluable reasoning process, enabling self-optimization of action knowledge without real-world interaction. Our method integrates LLM-based action modeling, procedural scene synthesis, Monte Carlo Tree Search (MCTS)-guided multi-step trajectory exploration, trajectory evaluation, and knowledge distillation. Evaluated across multiple novel environment benchmarks, it significantly improves action success rate and task completion rate, demonstrating effectiveness, robustness, and cross-domain generalizability. The code is publicly available.

Technology Category

Search and Optimization: Learning to SearchMachine Learning: Large Multimodal Models (LMMs)Multiagent Systems: Multiagent Learning

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

In the interaction between agents and their environments, agents expand their capabilities by planning and executing actions. However, LLM-based agents face substantial challenges when deployed in novel environments or required to navigate unconventional action spaces. To empower agents to autonomously explore environments, optimize workflows, and enhance their understanding of actions, we propose SynWorld, a framework that allows agents to synthesize possible scenarios with multi-step action invocation within the action space and perform Monte Carlo Tree Search (MCTS) exploration to effectively refine their action knowledge in the current environment. Our experiments demonstrate that SynWorld is an effective and general approach to learning action knowledge in new environments. Code is available at https://github.com/zjunlp/SynWorld.

Problem

• Enhance agent action knowledge in new environments
• Enable autonomous scenario synthesis for exploration
• Optimize workflows via MCTS-based action refinement

Innovation

• Synthesizes multi-step action scenarios
• Uses Monte Carlo Tree Search
• Refines action knowledge autonomously