Depth Estimators Are Implicit Neural Fields for 3D Scene Geometry Inpainting and Reconstruction

2026-07-11

AI Summary

Real-world 3D scenes often exhibit missing geometry, and existing depth estimation methods frequently produce reconstructions inconsistent with observed structures while demonstrating limited generalization on out-of-distribution data. To address these challenges, this work proposes Neural Depth Fields (NDF), the first approach to model depth estimation as a scene-level implicit neural field. NDF achieves geometric consistency and target-domain adaptation through a single test-time optimization step, eliminating the need for additional training while enabling high-quality completion of missing structures. The method yields high-fidelity, globally consistent 3D reconstructions across diverse scenes, reducing cross-view inconsistency by 63.3% and improving completion accuracy by 23.1%, thereby establishing state-of-the-art performance.

Abstract

The 3D geometry of real-world scene data is often incomplete. Mainstream methods use depth estimators to inpaint missing structure. However, their prediction results can be inconsistent with observed geometry, or unreliable on out-of-distribution data. To solve these problems, we propose Neural Depth Field (NDF). Our key insight is that a depth estimator can also be a scene-level implicit field. As an estimator, it adapts to the target domain by learning observed depth data. As an implicit field, it fits the existing geometry to maintain consistency. Under this view, NDF addresses both problems through a single test-time optimization. Experiments show that NDF produces high-fidelity and globally consistent geometry across diverse scene data, ranging from indoor scans to satellite imagery. It reduces cross-view inconsistency by 63.3\% and improves inpainting accuracy by 23.1\%, achieving state-of-the-art performance in 3D scene geometry inpainting. The code is available at: https://github.com/Shadow-Dream/Neural-Depth-Field.

Problem

• 3D scene geometry
• depth estimation
• geometry inpainting
• out-of-distribution data
• geometric consistency

Innovation

• Neural Depth Field
• implicit neural field
• depth estimation
• 3D geometry inpainting
• test-time optimization