PatchWorld: Gradient-Free Optimization of Executable World Models

2026-05-29

AI Summary

This work proposes a gradient-free and large language model–free approach for partially observable environments, leveraging counterexample-guided program synthesis and code repair to translate offline trajectories into executable Python world models. The resulting model replaces black-box predictions with symbolic belief-state programs, enabling inspectable, replayable, and locally patchable action-update mechanisms. Evaluated on seven AgentGym environments, the method—termed PatchWorld-Simple—achieves a macro success rate of 76.4%, outperforming baselines, and supports real-time single-step lookahead planning. This study presents the first fully executable-code-based world model, revealing a critical trade-off between observational fidelity and action discriminability.

Abstract

Text-agent environments are typically modeled as partially observable Markov decision processes (POMDPs), assuming that the simulator's latent state and transition dynamics are hidden from the agent. Yet little work has examined whether executable code can be induced to serve as a world model for prediction and planning under partial observability. We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models through counterexample-guided code repair. Instead of predicting the next observation with a black-box model, PatchWorld induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched. Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score among evaluated methods, reaching 76.4\% macro success in live one-step lookahead while invoking no LLM calls inside the world-model prediction module itself. We further find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility. This exposes a tradeoff in executable world models, since improving observation fidelity can come at the expense of action-discriminative dynamics, and vice versa. Code is available at https://github.com/HKBU-KnowComp/PatchWorld.

Problem

• executable world models
• partial observability
• code-based planning
• observation fidelity
• action-discriminative dynamics

Innovation

• gradient-free optimization
• executable world models
• counterexample-guided program synthesis
• symbolic belief-state programs
• code-based planning