3D-GIMP: When 3D Gaussian Inpainting Meets PatchMatch

2026-07-22

AI Summary

This work addresses the high computational cost, loss of fine details, and multi-view inconsistency—caused by “hallucination drift”—in existing diffusion-based 3D scene editing methods. The authors propose a hybrid paradigm that performs generative image inpainting only on a key reference view and propagates high-frequency textures to other views via a 3D-aware PatchMatch algorithm, thereby avoiding the stochasticity introduced by per-frame diffusion. By uniquely integrating single-view generative priors with 3D-aware PatchMatch, the method achieves mathematically consistent 3D reconstruction at arbitrary resolutions, significantly improving rendering speed and multi-view consistency while preserving inpainting quality on par with current diffusion-based approaches.

Abstract

Recent advances in 3D scene editing have leveraged iterative diffusion models to update input views. However, this process is computationally expensive and struggles to produce sharp details. Meanwhile, ``hallucination drift'' frequently introduces multi-view inconsistencies, leading to structural artifacts when rendering novel viewpoints. To address this problem, we present 3D-GIMP (3D Gaussian Inpainting Meets Patch Matching), a novel hybrid paradigm designed for high-fidelity object removal in 3D Gaussian Splatting. Instead of diffusing every view, 3D-GIMP performs a single generative inpainting on a key reference view, which serves as an appearance prior. We then introduce a 3D-aware PatchMatch algorithm to propagate these reference textures across all remaining views via correspondence matching, effectively bypassing the stochastic nature of frame-by-frame diffusion. By prioritizing reconstructive consistency over iterative generation, 3D-GIMP maintains high-frequency details across arbitrary resolutions while ensuring a mathematically consistent 3D reconstruction. Our experiments demonstrate that 3D-GIMP not only achieves competitive inpainting quality as previous methods using diffusion in multiple views, but also outperforms these methods in rendering speed and view consistency.

Problem

• 3D scene editing
• multi-view inconsistency
• hallucination drift
• 3D Gaussian Splatting
• object removal

Innovation

• 3D Gaussian Splatting
• Generative Inpainting
• PatchMatch
• View Consistency
• 3D Scene Editing