Multiview Geometric Regularization of Gaussian Splatting for Accurate Radiance Fields

2025-06-16

AI Summary

To address inaccurate geometry reconstruction and surface artifacts in 3D Gaussian Splatting (3DGS) under multi-view scenes with significant color discrepancies, this paper proposes a multi-view stereo (MVS)-guided joint geometry-appearance optimization framework. Methodologically, it introduces the first MVS-driven Gaussian initialization strategy; incorporates a median depth relative loss with uncertainty-aware weighting; and jointly enforces MVS depth, normal, and RGB consistency constraints to model geometric complementarity between MVS priors and Gaussian optimization. Experiments across diverse indoor and outdoor scenes demonstrate substantial improvements in surface completeness and boundary sharpness—reducing Chamfer distance by 21.3%—while preserving high-fidelity rendering quality (surpassing state-of-the-art methods in PSNR and SSIM). This work establishes the first unified end-to-end co-optimization framework integrating MVS priors with 3DGS, enabling synergistic geometric refinement and appearance learning.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recent methods, such as 2D Gaussian Splatting and Gaussian Opacity Fields, have aimed to address the geometric inaccuracies of 3D Gaussian Splatting while retaining its superior rendering quality. However, these approaches still struggle to reconstruct smooth and reliable geometry, particularly in scenes with significant color variation across viewpoints, due to their per-point appearance modeling and single-view optimization constraints. In this paper, we propose an effective multiview geometric regularization strategy that integrates multiview stereo (MVS) depth, RGB, and normal constraints into Gaussian Splatting initialization and optimization. Our key insight is the complementary relationship between MVS-derived depth points and Gaussian Splatting-optimized positions: MVS robustly estimates geometry in regions of high color variation through local patch-based matching and epipolar constraints, whereas Gaussian Splatting provides more reliable and less noisy depth estimates near object boundaries and regions with lower color variation. To leverage this insight, we introduce a median depth-based multiview relative depth loss with uncertainty estimation, effectively integrating MVS depth information into Gaussian Splatting optimization. We also propose an MVS-guided Gaussian Splatting initialization to avoid Gaussians falling into suboptimal positions. Extensive experiments validate that our approach successfully combines these strengths, enhancing both geometric accuracy and rendering quality across diverse indoor and outdoor scenes.

Problem

• Improve geometric accuracy in 3D Gaussian Splatting
• Reconstruct smooth geometry in color-varying scenes
• Integrate multiview constraints for better depth estimation

Innovation

• Multiview geometric regularization for Gaussian Splatting
• MVS depth and normal constraints integration
• Median depth-based multiview relative depth loss