Benchmarking Single-Step Inpainting Methods for Multi-Object 3D Gaussian Splatting Scenes

2026-05-29

AI Summary

This work addresses the challenge of maintaining cross-view 3D consistency during object removal and inpainting in multi-object 3D Gaussian Splatting (3DGS) scenes. The authors propose a single-step optimization framework that integrates 2D image inpainting with 3DGS. Through systematic comparison of two strategies—reinitializing Gaussians from scratch versus fine-tuning existing parameters—and evaluation of both reconstruction-based and generative diffusion-based inpainting models, they find that reconstruction-based approaches better preserve geometric consistency. The study introduces the first multi-object 3DGS benchmark dataset featuring realistic occlusions and ground-truth annotations. Experimental results demonstrate that reinitialization significantly outperforms fine-tuning, achieving superior performance in both inpainting quality and 3D consistency, thereby highlighting the critical importance of a “remove-then-inpaint” paradigm for 3D scene editing.

Abstract

The tasks of object removal and inpainting 3D Gaussian Splatting (3DGS) scenes face challenges such as 3D consistency across camera views. In comparing 2D inpainters and their suitability for the 3D domain, we find that reconstruction-based inpainters outperform generative diffusion models in 3D consistency. Integrating these 2D inpainters into different single-step methods for creating and finetuning 3DGS scenes, our results indicate that initializing the scene from scratch produces higher quality results than finetuning the existing scene. Using a state-of-the-art generative 2D inpainter, we create a straightforward baseline to underline the importance of object removal before inpainting in the 3D setting. Since 360° datasets rarely include real-world ground truths, and challenging occlusion scenarios are equally sparse, we introduce a novel multi-object scene with recorded ground truth data and many views with object occlusions.

Problem

• 3D Gaussian Splatting
• inpainting
• object removal
• 3D consistency
• multi-object scenes

Innovation

• 3D Gaussian Splatting
• single-step inpainting
• 3D consistency
• object removal
• multi-object dataset