Technology Category
Computer Vision: 3D Computer VisionMachine Learning: Learning on the Edge & Model CompressionSearch and Optimization: Sampling/Simulation-based Search
Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training
🤖 AI Summary
This work addresses the prohibitively high peak memory consumption of 3D Gaussian Splatting (3DGS) during training, which stems from uncontrolled densification and hinders deployment on edge devices. To overcome this limitation, the authors propose a memory-constrained training framework that maintains near-constant low memory usage by dynamically balancing Gaussian growth and pruning throughout optimization. The key innovation lies in an adaptive Gaussian compensation mechanism that effectively harmonizes densification and pruning, surpassing the capabilities of existing post-processing approaches. Experimental results across diverse real-world scenes demonstrate that the method significantly outperforms current techniques, enabling direct training on a Jetson AGX Xavier with an 80% reduction in peak memory while preserving rendering quality comparable to the original 3DGS.
Technology Category
Application Category
📝 Abstract
3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gaussian primitives. However, it suffers from a substantial memory footprint, particularly during training due to uncontrolled densification, posing a critical bottleneck for deployment on memory-constrained edge devices. While existing methods prune redundant Gaussians post-training, they fail to address the peak memory spikes caused by the abrupt growth of Gaussians early in the training process. To solve the training memory consumption problem, we propose a systematic memory-bounded training framework that dynamically optimizes Gaussians through iterative growth and pruning. In other words, the proposed framework alternates between incremental pruning of low-impact Gaussians and strategic growing of new primitives with an adaptive Gaussian compensation, maintaining a near-constant low memory usage while progressively refining rendering fidelity. We comprehensively evaluate the proposed training framework on various real-world datasets under strict memory constraints, showing significant improvements over existing state-of-the-art methods. Particularly, our proposed method practically enables memory-efficient 3DGS training on NVIDIA Jetson AGX Xavier, achieving similar visual quality with up to 80% lower peak training memory consumption than the original 3DGS.
Problem
Research questions and friction points this paper is trying to address.
3D Gaussian Splatting
memory consumption
training memory
memory-constrained devices
densification
Innovation
Methods, ideas, or system contributions that make the work stand out.
memory-bounded training
3D Gaussian Splatting
dynamic pruning and growth
adaptive Gaussian compensation
edge deployment
🔎 Similar Papers
No similar papers found.
