FLoD: Integrating Flexible Level of Detail into 3D Gaussian Splatting for Customizable Rendering

2024-08-23

arXiv.org

AI Summary

Existing 3D Gaussian Splatting (3DGS) methods suffer from poor hardware adaptability: lightweight variants compromise reconstruction quality, while high-fidelity approaches demand excessive GPU memory, hindering deployment across heterogeneous devices. To address this, we propose Flexible Levels of Detail (FLoD), the first scalable LoD architecture explicitly designed for 3DGS—enabling single-model, multi-granularity reconstruction and dynamic Gaussian count adjustment. Our method integrates differentiable Gaussian parameterization, adaptive density control, hierarchical scene representation, and real-time rendering optimization to achieve fine-grained trade-offs between visual quality and memory consumption. FLoD is framework-agnostic and compatible with mainstream systems including Instant-NGP and GaussianMamba. On resource-constrained devices, it reduces memory usage by up to 67% while maintaining PSNR > 28 dB. To our knowledge, this is the first work enabling high-quality real-time 3DGS rendering across diverse hardware platforms.

Technology Category

Computer Vision: 3D Computer VisionMachine Learning: Hardware-aware MLSearch and Optimization: Mixed Discrete/Continuous Search

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsUser Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

3D Gaussian Splatting (3DGS) achieves fast and high-quality renderings by using numerous small Gaussians, which leads to significant memory consumption. This reliance on a large number of Gaussians restricts the application of 3DGS-based models on low-cost devices due to memory limitations. However, simply reducing the number of Gaussians to accommodate devices with less memory capacity leads to inferior quality compared to the quality that can be achieved on high-end hardware. To address this lack of scalability, we propose integrating a Flexible Level of Detail (FLoD) to 3DGS, to allow a scene to be rendered at varying levels of detail according to hardware capabilities. While existing 3DGSs with LoD focus on detailed reconstruction, our method provides reconstructions using a small number of Gaussians for reduced memory requirements, and a larger number of Gaussians for greater detail. Experiments demonstrate our various rendering options with tradeoffs between rendering quality and memory usage, thereby allowing real-time rendering across different memory constraints. Furthermore, we show that our method generalizes to different 3DGS frameworks, indicating its potential for integration into future state-of-the-art developments. Project page: https://3dgs-flod.github.io/flod.github.io/

Problem

• 3DGS lacks flexibility for varying hardware setups
• Existing methods compromise quality or require high-end GPUs
• No adaptable Level of Detail solution for 3DGS

Innovation

• Multi-level 3DGS representation with scale constraints
• Level-by-level training for structural consistency
• Selective rendering at varying detail levels