GP-GS: Gaussian Processes for Enhanced Gaussian Splatting

2025-02-04

AI Summary

3D Gaussian Splatting (3DGS) suffers from degraded reconstruction quality due to reliance on sparse Structure-from-Motion (SfM) point clouds. To address this, we propose Gaussian Process Gaussian Splatting (GP-GS), an adaptive point cloud densification framework built upon multi-output Gaussian processes (MOGPs). Our method introduces two key innovations: (1) uncertainty-guided, depth-map-driven dynamic sampling that infers geometrically consistent candidate points from 2D pixel locations and predicted depth; and (2) a variance-driven pruning mechanism for high-fidelity Gaussian initialization. GP-GS operates without additional supervision, significantly enhancing both the density and geometric fidelity of the initial Gaussian distribution. Evaluated on synthetic and real-world datasets, GP-GS consistently outperforms state-of-the-art baselines in PSNR and SSIM, establishing a more robust and accurate initialization paradigm for 3DGS.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

3D Gaussian Splatting has emerged as an efficient photorealistic novel view synthesis method. However, its reliance on sparse Structure-from-Motion (SfM) point clouds consistently compromises the scene reconstruction quality. To address these limitations, this paper proposes a novel 3D reconstruction framework Gaussian Processes Gaussian Splatting (GP-GS), where a multi-output Gaussian Process model is developed to achieve adaptive and uncertainty-guided densification of sparse SfM point clouds. Specifically, we propose a dynamic sampling and filtering pipeline that adaptively expands the SfM point clouds by leveraging GP-based predictions to infer new candidate points from the input 2D pixels and depth maps. The pipeline utilizes uncertainty estimates to guide the pruning of high-variance predictions, ensuring geometric consistency and enabling the generation of dense point clouds. The densified point clouds provide high-quality initial 3D Gaussians to enhance reconstruction performance. Extensive experiments conducted on synthetic and real-world datasets across various scales validate the effectiveness and practicality of the proposed framework.

Problem

• Enhance 3D Gaussian Splatting reconstruction
• Densify sparse SfM point clouds adaptively
• Ensure geometric consistency using uncertainty estimates

Innovation

• Gaussian Process densification
• dynamic sampling pipeline
• uncertainty-guided pruning