Bridging the Agent-World Gap: Text World Models for LLM-based Agents

2026-06-08

AI Summary

Current large language model (LLM)-based agents lack explicit modeling of textual environment dynamics, limiting their planning and learning capabilities. This work proposes Textual World Models (TWMs), which predict the textual state transitions induced by actions, thereby enabling agents to synthesize experiences, plan, and adapt during both training and inference. We present the first systematic theoretical framework for TWMs, encompassing state representation, construction paradigms—such as LLM-as-WM and code-as-WM—application mechanisms, and an evaluation protocol, clarifying the design space and highlighting key challenges for future research. Empirical results demonstrate that TWMs substantially enhance the efficiency and robustness of LLM agents in long-horizon interactive tasks, including web navigation and tool invocation.

Abstract

Large language model (LLM)-based agents are increasingly used in interactive textual environments, from web navigation and code editing to tool use and long-horizon dialogue. Yet many remain largely reactive, mapping observations to actions without an explicit model of how these environments are structured and evolve. This motivates text world models (TWMs): transition models over textual states that, given a state and a candidate action, predict the resulting webpage, terminal output, API response, or user reply, thereby supporting planning, efficient learning, and principled evaluation. We systematically review text world models for LLM-based agents, organized around a formal framework and the agent lifecycle: (1) Foundations, defining text world models and characterizing them by state representation and grounding domain; (2) Construction, taxonomizing LLM-as-WM and code-as-WM paradigms and reviewing methods for building them; (3) Application, examining how world models support agents at training time through experience synthesis and at inference time through planning, verification, and adaptation; and (4) Evaluation, covering both evaluation of the world model itself and its use as an evaluation environment for agents. We aim to consolidate this rapidly developing area, clarify its design space, and highlight open challenges for future research.

Problem

• text world models
• LLM-based agents
• interactive textual environments
• transition models
• agent-world gap

Innovation

• Text World Models
• LLM-based Agents
• Transition Models
• Experience Synthesis
• Planning