PCAsplat: Gaussian Splatting with Local PCA Regularization

2026-10-07

AI Summary

This study addresses the issue of floating artifacts in 3D Gaussian Splatting, which arise when occluded or weakly contributing Gaussians drift due to rasterization losses. To mitigate this, we propose a geometry-aware regularization framework based on differentiable local Principal Component Analysis (PCA). Moving beyond conventional view-dependent approaches, our method directly optimizes the neighborhood structure of Gaussian centers by constraining PCA eigenvalues and enforcing normal alignment, thereby achieving geometric correction even for Gaussians lacking gradient contributions. Experimental results demonstrate that the proposed approach significantly suppresses floating noise and enhances surface reconstruction quality. Furthermore, it effectively facilitates downstream tasks such as scene segmentation and Poisson surface reconstruction.

Abstract

Gaussian splatting has emerged as a flexible representation for 3D reconstruction from posed images. However, existing methods are optimized primarily using rasterization-based losses, which supervise a splat only when it contributes to sampled camera rays. Gaussians that are occluded or contribute little to the sampled view therefore receive weak or no geometric gradients and may drift away from the underlying surface, producing undesired floaters. We introduce PCAsplat, a geometry-aware regularization framework for Gaussian splatting based on differentiable local principal component analysis (PCA). Our PCA regularizer acts directly on neighborhoods of Gaussian centers and can therefore update Gaussians that do not contribute to the current training view. We regularize the PCA eigenvalues to encourage Gaussians to move to the underlying surface with isotropic tangent-plane coverage. We also align each Gaussian normal with the PCA-estimated neighborhood normal to enforce consistent orientation. Experiments on DTU, Tanks and Temples, and NeRF Synthetic show that the splats produced by PCAsplat better approximate samples of the reference surface while substantially reducing undesired floaters. These surface-aligned splats enable downstream geometry-processing tasks, including point cloud segmentation, and direct Poisson reconstruction. Additionally, PCAsplat remains competitive under conventional novel view synthesis and mesh extraction tasks. Code will be released.

Problem

• Gaussian Splatting
• 3D Reconstruction
• Floaters
• Geometric Regularization
• Surface Alignment

Innovation

• Gaussian Splatting
• Local PCA Regularization
• 3D Reconstruction
• Geometry-aware
• Surface Alignment