WorldAgent: Verification-Guided Agentic Physical World Construction

2026-09-27

AI Summary

This study addresses the challenge of constructing complex physical worlds from natural language, where multi-scale environmental coordination and implicit physical constraints are difficult to satisfy automatically. To overcome this, we propose a verification-guided agent framework that translates natural language prompts into structured specifications and introduces a verification layer to inspect scene geometry and simulation states in real time. Through closed-loop iteration, the system automatically rectifies failures and re-executes tasks, enabling the construction of physical worlds without manual debugging. Experimental results demonstrate that our approach achieves leading performance across five metrics on the AgenticSimBench benchmark and receives the highest ratings across all four criteria in user studies.

Abstract

Constructing complex physical worlds from language requires coordinating extensive 3D environments, detailed structures and objects at different spatial scales, and interacting physical processes under both stated goals and implicit physical constraints. We present WorldAgent, an agentic framework for verification-guided physical world construction from a single natural-language prompt, without iterative user debugging. A world construction layer expands the prompt into a structured world specification and uses physical knowledge to build scenes and run numerical simulations. After every step, a verification layer inspects scene geometry and simulation states alongside rendered views. Failed checks guide automatic revisions to the specification and re-execution of the affected steps. Accepted worlds pass the required checks and remain editable for further inspection and resimulation. We introduce AgenticSimBench, on which WorldAgent achieves the best scores among the evaluated agent-based methods on five of seven metrics. In a 26-participant user study, it receives the highest mean ratings across all four criteria.

Problem

• Physical World Construction
• Natural Language Prompt
• 3D Environment
• Physical Simulation
• Agentic Framework

Innovation

• Agentic Framework
• Verification-Guided Construction
• Physical World Simulation
• Self-Correction
• AgenticSimBench