World Observer: Joint Actor-Observer Generation for Persistent World Modeling

2026-10-01

AI Summary

This study addresses the loss of off-screen object states in video world models caused by limited fields of view. To overcome this, we propose a novel framework that jointly generates actors and panoramic observers. By decoupling observation from action and introducing an observer mechanism alongside an observer pool, our approach enables the continuous evolution and high-resolution recovery of off-screen objects. Furthermore, the method incorporates shared panoramic source geometric correction, multi-view joint generation, and spatial metric evaluation strategies. Experimental results demonstrate that the proposed framework significantly enhances off-screen dynamic consistency while preserving visual fidelity and controllability, effectively resolving the challenge of modeling out-of-view states in video world models.

Abstract

How can a world model continuously observe regions beyond the actor's current view? Video world models simulate how an environment evolves from an agent's actions, yet remain actor-centric. Once an object leaves the actor's view, they lose direct evidence of its evolution, often failing to preserve its state and dynamics upon re-entry. To address this, we introduce World Observer, which decouples observing from acting by jointly generating a perspective actor for the agent-centric view with one or more panoramic observers that watch selected world regions. This allows objects that leave the actor's view to remain visually evolving in an observer, so their updated states are reflected when they re-enter. We ground the actor and observers by warping from a shared panoramic source for explicit geometric correspondence, and introduce an Observer Sink of high-resolution perspective references to restore fine appearance upon re-entry. Since the observers are decoupled from the actor, they can be placed freely across the scene, extended to multiple locations for broader coverage, and driven by control signals to steer out-of-view evolution. To evaluate out-of-view evolution, we further introduce world-space metrics and a benchmark spanning real and synthetic scenes. World Observer substantially improves out-of-view dynamics while remaining competitive in visual fidelity, camera control, and 3D adherence.

Problem

• world model
• out-of-view evolution
• persistent world modeling
• video generation

Innovation

• World Model
• Actor-Observer Decoupling
• Panoramic Generation
• Out-of-view Dynamics
• Geometric Grounding