GaussianUDF: Inferring Unsigned Distance Functions through 3D Gaussian Splatting

2025-03-25

AI Summary

Existing 3D Gaussian Splatting (3DGS) struggles to reconstruct open surfaces from multi-view images due to incompatibility between its discrete, explicit representation and the continuous, implicit modeling of Unsigned Distance Functions (UDFs). Method: This paper proposes the first joint 3DGS-UDF learning framework. It introduces a differentiable 2D Gaussian plane parameterization to represent local surface geometry, enables UDF gradient estimation via self-supervised gradient inference, and imposes a zero-level-set neighborhood constraint to stabilize optimization. The framework is fully end-to-end differentiable and requires no depth maps, masks, or voxel supervision. Results: Evaluated on standard benchmarks and real-world scenes, the method significantly improves reconstruction completeness, boundary sharpness, and geometric accuracy, while maintaining efficient training and real-time rendering. It achieves state-of-the-art performance across all major metrics, outperforming existing approaches.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Reconstructing open surfaces from multi-view images is vital in digitalizing complex objects in daily life. A widely used strategy is to learn unsigned distance functions (UDFs) by checking if their appearance conforms to the image observations through neural rendering. However, it is still hard to learn continuous and implicit UDF representations through 3D Gaussians splatting (3DGS) due to the discrete and explicit scene representation, i.e., 3D Gaussians. To resolve this issue, we propose a novel approach to bridge the gap between 3D Gaussians and UDFs. Our key idea is to overfit thin and flat 2D Gaussian planes on surfaces, and then, leverage the self-supervision and gradient-based inference to supervise unsigned distances in both near and far area to surfaces. To this end, we introduce novel constraints and strategies to constrain the learning of 2D Gaussians to pursue more stable optimization and more reliable self-supervision, addressing the challenges brought by complicated gradient field on or near the zero level set of UDFs. We report numerical and visual comparisons with the state-of-the-art on widely used benchmarks and real data to show our advantages in terms of accuracy, efficiency, completeness, and sharpness of reconstructed open surfaces with boundaries. Project page: https://lisj575.github.io/GaussianUDF/

Problem

• Reconstruct open surfaces from multi-view images
• Bridge gap between 3D Gaussians and UDFs
• Improve accuracy and efficiency of UDF learning

Innovation

• Overfitting thin 2D Gaussian planes on surfaces
• Self-supervision and gradient-based UDF inference
• Novel constraints for stable 2D Gaussian optimization