Optimize the Unseen - Fast NeRF Cleanup with Free Space Prior

2024-12-17

arXiv.org

AI Summary

NeRF’s photometric reconstruction often suffers from “floating artifacts” in unobserved regions, severely degrading novel-view synthesis quality. To address this, we propose a maximum a posteriori (MAP)-based artifact suppression method that introduces, for the first time, a concise and global free-space prior—replacing conventional maximum-likelihood or local, data-driven priors. Our approach uniformly suppresses artifacts in both visible and occluded regions without requiring auxiliary networks or additional GPU memory. It preserves geometric and appearance fidelity within observed regions while completing cleanup in under 30 seconds. The method accelerates inference by 2.5×, incurs zero extra memory overhead, and significantly improves novel-view image fidelity and geometric consistency—especially under challenging sparse-view settings.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Learning with Manifolds
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Neural Radiance Fields (NeRF) have advanced photorealistic novel view synthesis, but their reliance on photometric reconstruction introduces artifacts, commonly known as"floaters". These artifacts degrade novel view quality, especially in areas unseen by the training cameras. We present a fast, post-hoc NeRF cleanup method that eliminates such artifacts by enforcing our Free Space Prior, effectively minimizing floaters without disrupting the NeRF's representation of observed regions. Unlike existing approaches that rely on either Maximum Likelihood (ML) estimation to fit the data or a complex, local data-driven prior, our method adopts a Maximum-a-Posteriori (MAP) approach, selecting the optimal model parameters under a simple global prior assumption that unseen regions should remain empty. This enables our method to clean artifacts in both seen and unseen areas, enhancing novel view quality even in challenging scene regions. Our method is comparable with existing NeRF cleanup models while being 2.5x faster in inference time, requires no additional memory beyond the original NeRF, and achieves cleanup training in less than 30 seconds. Our code will be made publically available.

Problem

• Eliminating floater artifacts in NeRF synthesis
• Improving novel view quality in unseen areas
• Fast cleanup without disrupting observed regions

Innovation

• Uses Maximum-a-Posteriori with Free Space Prior
• Eliminates floaters in seen and unseen regions
• Achieves faster inference without extra memory