Code Plans, Diffusion Renders: Open-Ended Generative World Modeling

2026-09-22

AI Summary

为了解决世界建模中的长期记忆与开放互动问题，CoDeR通过代码构建可执行的世界，并利用视频生成模型实现视觉效果。

Abstract

We introduce \textbf{CoDeR}, a new paradigm for world modeling. Unlike existing video world models that implicitly represent world dynamics through visual observations, our system explicitly constructs an executable world with code and employs video generation models for visual realization. Specifically, we coordinate five complementary roles to translate high-level concepts into structured world rules, executable dynamics, and perceptual observations. This design enables \textit{long-term memory}, \textit{open-ended interactions}, \textit{autonomous world evolution}, and \textit{multi-agent scenarios}, where multiple entities can act, interact, and evolve persistently beyond the current observation. Extensive experiments demonstrate that our framework substantially extends the capabilities of existing world models, enabling long-term memory, open-ended interactions, autonomous evolution, and persistent multi-agent dynamics, while achieving state-of-the-art performance across multiple evaluation settings. Code and model weights will be made publicly available. Project Page: \href{https://becauseimbatman0.github.io/CoDeR}{CoDeR}.

Problem

• world modeling
• long-term memory
• open-ended interactions
• autonomous world evolution
• multi-agent scenarios

Innovation

• World Modeling
• Executable Code
• Video Generation
• Open-Ended Interactions
• Long-Term Memory