Reconstructing the Dynamic World: A Representation-Centric View of 4D Scene Reconstruction

2026-09-30

AI Summary

This study addresses the fragmentation in dynamic scene reconstruction caused by non-rigid motion, occlusion, and efficiency trade-offs by proposing a representation-centric unified framework. The work systematically reviews mainstream approaches, including Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), analyzing how design choices along the representation and temporal modeling dimensions affect reconstruction quality and computational efficiency. Furthermore, it consolidates existing datasets and evaluation metrics to expose current experimental limitations and identify promising future research directions. Ultimately, this project establishes a structured theoretical foundation for understanding dynamic scene reconstruction and provides an open-source, continuously updated repository of supporting resources.

Abstract

4D scene reconstruction aims to recover the evolving geometry, appearance, and motion of dynamic environments from visual observations. Despite substantial progress in neural scene representations, reconstructing dynamic scenes remains challenging due to non-rigid motion, occlusions, temporal inconsistencies, and the trade-offs between reconstruction fidelity and computational efficiency. Recent advances in Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have introduced diverse approaches to representing and reconstructing dynamic scenes, yet their relationships, underlying design choices, and evaluation protocols remain fragmented. In this paper, we present a unified perspective on 4D scene reconstruction, organizing existing methods around their scene representations, temporal modeling strategies, reconstruction pipelines, and optimization objectives. Through this framework, we examine how different design choices affect geometric fidelity, appearance consistency, motion representation, and computational efficiency. We further consolidate commonly used datasets and evaluation metrics, identify limitations in current experimental practices, and discuss open challenges in reconstructing complex, dynamic real-world environments. By connecting methodological developments with their underlying assumptions and evaluation evidence, this work provides a structured foundation for understanding existing approaches and identifying future research directions. An evolving collection of relevant papers and resources is available at https://github.com/ZiyangYan/Awesome-4D-Scene-Reconstruction.

Problem

• 4D scene reconstruction
• dynamic scenes
• scene representation
• neural radiance fields
• 3D Gaussian Splatting

Innovation

• 4D Scene Reconstruction
• Scene Representation
• Neural Radiance Fields
• 3D Gaussian Splatting
• Temporal Modeling