Spackle: Completing Large View Single Image NVS with Adaptive Gaussians

2026-09-25

AI Summary

This study addresses the capacity competition between visible and occluded regions in single-image novel view synthesis, which arises from using a fixed number of Gaussians. To mitigate this issue, we propose a lightweight residual learning framework based on 3D Gaussian Splatting. The method operates in three stages: predicting baseline attributes, identifying reconstruction deficiencies to optimize residual Gaussians, and merging both components during inference for efficient rendering. By introducing an adaptive residual mechanism, our approach effectively alleviates capacity competition without increasing computational overhead. It significantly enhances scene fidelity under large viewpoint deviations, achieving state-of-the-art performance.

Abstract

Single-image novel view synthesis (NVS) enables photorealistic rendering of un- observed viewpoints from a single input. Practical NVS systems require two key capabilities: robust reconstruction of occluded regions and high inference effi- ciency. While hybrid decoupled frameworks combining feedforward 3D Gaussian Splatting (3DGS) and diffusion models show promise for large-view-deviation NVS, they suffer from capacity competition: a fixed number of Gaussians forces resource shifts from visible to newly disoccluded areas, degrading original scene fidelity when the target view deviates significantly from the input. To address this, we propose Spackle, a lightweight residual learning framework that mit- igates capacity competition without sacrificing efficiency. Spackle operates in three stages: predicting base 3DGS attributes from given views, automatically identifying poorly reconstructed regions, and learning a residual 3DGS optimized exclusively for these areas. At inference, we combine the baseline and aug- mented Gaussians for NVS. We conduct comprehensive experiments and show that Spackle achieves state-of-the-art performance on large-view-deviation cases.

Problem

• Novel View Synthesis
• Single Image
• Large View Deviation
• Capacity Competition
• 3D Gaussian Splatting

Innovation

• Novel View Synthesis
• 3D Gaussian Splatting
• Residual Learning
• Capacity Competition
• Large-View-Deviation