ReSplat: Learning Recurrent Gaussian Splats

2025-10-09

AI Summary

Existing feedforward Gaussian splatting models are constrained by single-pass forward inference, preventing iterative optimization of 3D Gaussian parameters and thus limiting generalization and rendering fidelity. To address this, we propose Feedforward Cyclic Gaussian Splatting (FCGS), a novel framework incorporating a lightweight cyclic mechanism that refines Gaussian positions, scales, and opacities across successive iterations using rendering error as a gradient-free feedback signal. Our method integrates low-resolution initialization, subsampled spatial reconstruction, and error-driven parameter updates, drastically reducing the initial number of Gaussians. Extensive experiments across multi-view, multi-resolution, and cross-dataset settings demonstrate that FCGS achieves state-of-the-art rendering quality with significantly fewer Gaussians—improving inference speed by 2.1× over prior methods—while exhibiting strong adaptive generalization to unseen scenes.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

While feed-forward Gaussian splatting models provide computational efficiency and effectively handle sparse input settings, their performance is fundamentally limited by the reliance on a single forward pass during inference. We propose ReSplat, a feed-forward recurrent Gaussian splatting model that iteratively refines 3D Gaussians without explicitly computing gradients. Our key insight is that the Gaussian splatting rendering error serves as a rich feedback signal, guiding the recurrent network to learn effective Gaussian updates. This feedback signal naturally adapts to unseen data distributions at test time, enabling robust generalization. To initialize the recurrent process, we introduce a compact reconstruction model that operates in a $16 imes$ subsampled space, producing $16 imes$ fewer Gaussians than previous per-pixel Gaussian models. This substantially reduces computational overhead and allows for efficient Gaussian updates. Extensive experiments across varying of input views (2, 8, 16), resolutions ($256 imes 256$ to $540 imes 960$), and datasets (DL3DV and RealEstate10K) demonstrate that our method achieves state-of-the-art performance while significantly reducing the number of Gaussians and improving the rendering speed. Our project page is at https://haofeixu.github.io/resplat/.

Problem

• Improves Gaussian splatting by adding iterative refinement without gradients
• Uses rendering error as feedback for robust generalization on unseen data
• Reduces computational overhead with compact initialization and fewer Gaussians

Innovation

• Recurrent Gaussian splatting model refines 3D Gaussians iteratively
• Uses rendering error as feedback signal for Gaussian updates
• Initializes process with compact reconstruction in subsampled space