NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Models

2026-09-21

AI Summary

为解决物理信息4D生成依赖手动假设问题，提出NeuIDO框架，通过从视觉观察学习统一内在动力学表示，实现零样本动力学推断。

Abstract

World models aim to capture environmental dynamics and predict future trajectories, showing growing potential for embodied intelligence. Physics-informed 4D generation integrates physical simulation to predict 3D object interactions, offering a promising pathway toward world models. However, this paradigm relies on manually imposed dynamical assumptions rather than internalizing world dynamics, and thus still leaves a gap toward a true world model. To bridge this gap, we propose NeuIDO, a novel world dynamics modeling framework that learns a unified intrinsic dynamics representation from visual observations, advancing physics-informed 4D generation toward a world model. Specifically, we formulate world modeling as a neural operator learning problem and introduce a two-stage training strategy to learn a generalizable mapping from the visual observation distribution to the intrinsic dynamics distribution. Building on this observation-dynamics mapping, NeuIDO enables zero-shot dynamics inference directly from videos and can be further aligned with complex real-world dynamics via few-shot adaptation. Extensive experiments demonstrate that NeuIDO effectively unifies the intrinsic dynamics underlying diverse visual observations into a shared representation and rapidly infers dynamics in novel scenes.

Problem

• world models
• intrinsic dynamics
• physics-informed 4D generation
• visual observations

Innovation

• NeuIDO
• Intrinsic Dynamics
• Physics-Informed 4D Generation
• Zero-Shot Dynamics Inference