GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis

2026-07-23

AI Summary

Dynamic scene reconstruction faces the challenge of simultaneously achieving fine-grained motion modeling, structural stability, and compact representation. This work proposes GrainGS, a novel framework that constructs a hierarchical anchor-based canonical space to disentangle static structure from dynamics. It models geometric deformation via per-Gaussian warping and decomposes appearance into canonical and residual components to capture photometric variations. A key innovation is a gradient decoupling mechanism that, during joint training, blocks the deformation path from updating canonical positions while still allowing direct optimization toward the reconstruction objective. This enables precise local motion while preserving global structural constraints. Experiments demonstrate high-quality reconstructions on both synthetic monocular and real multi-view datasets, achieving a PSNR of 36.98 dB, rendering at 435.6 FPS, and requiring only 4.67 MB of storage.

Abstract

Dynamic scene reconstruction with 3D Gaussian Splatting requires a balance between fine-grained motion modeling, structural stability, and compact representation. Existing per-primitive methods provide flexible local deformation but often suffer from redundant primitive growth, while anchor-based methods improve spatial regularity at the cost of suppressing locally varying motion. To address these issues, we present GrainGS, a dynamic Gaussian framework that combines a hierarchical anchor scaffold with per-Gaussian deformation. A static warm-up stage first establishes a time-invariant canonical representation from observations across all timestamps. During joint training, a stop-gradient operation blocks the deformation-mediated gradient pathway to the canonical positions while preserving their direct refinement through the reconstruction objective. Each Gaussian then predicts independent temporal offsets for position, rotation, and scale, enabling detailed local motion within a structurally constrained scaffold. A canonical-residual appearance decomposition further models frame-dependent photometric changes without forcing them into geometric deformation. Experiments on synthetic monocular and real-world multiview benchmarks show that GrainGS achieves high reconstruction quality, real-time novel view synthesis, and compact storage. Under the synthetic benchmark setting, it reaches an average peak signal-to-noise ratio of 36.98 decibels, renders at 435.6 frames per second, and requires 4.67 megabytes of storage.

Problem

• Dynamic Scene Reconstruction
• 3D Gaussian Splatting
• Novel View Synthesis
• Motion Modeling
• Compact Representation

Innovation

• Gradient-Decoupled
• Gaussian Splatting
• Dynamic Scene Reconstruction
• Canonical-Residual Decomposition
• Stop-Gradient