StreamSplat: Streaming Feed-Forward 3D Gaussian Splatting

2026-08-02

AI Summary

This work addresses the limitation of existing feed-forward 3D Gaussian splatting methods, which rely on fixed viewpoint sets and struggle to adapt to online scenarios where views arrive sequentially. The authors propose a streaming feed-forward 3D Gaussian splatting framework that incrementally maintains a geometrically consistent scene representation, enabling real-time generation of renderable Gaussian models upon arrival of each new batch of views. Key innovations include Voxel-Aligned Causal Caching (VACC) for memory-efficient long-term history retention, Historical Projected Depth Anchoring (HPDA), and Cache-Guided Feature Injection (CGFI) to enhance causal prediction accuracy. Experiments demonstrate competitive novel view synthesis performance on DL3DV, RealEstate10K, and ScanNet using sparse causal inputs, with successful scaling to sequences of up to 1,024 views while continuously improving rendering quality.

Abstract

Feed-forward 3D Gaussian Splatting enables efficient novel-view synthesis without per-scene optimization, but most existing methods assume a fixed set of context views and process them jointly. This limits their applicability to online scenarios where calibrated views arrive sequentially and the scene must be updated causally. We present \emph{StreamSplat}, a streaming feed-forward 3DGS framework that incrementally maintains a persistent geometry-grounded scene state and decodes it into renderable 3D Gaussians after each input chunk. StreamSplat centers on a \textbf{Voxel-Aligned Causal Cache (VACC)}, which stores historical 3D tokens in a memory-bounded voxel structure so that memory grows with explored scene geometry rather than stream length. To better reuse history during causal prediction, we introduce \textbf{History-Projected Depth Anchoring (HPDA)} to project cached geometry as depth guidance for current cost-volume estimation, and \textbf{Cache-Guided Feature Injection (CGFI)} to inject cached latent evidence into Gaussian-token regression. Experiments on DL3DV, RealEstate10K, and ScanNet show that StreamSplat remains competitive with state-of-the-art feed-forward 3DGS methods under sparse causal inputs, despite not using future views or full-scene context. More importantly, it scales to long input streams with 256, 512, and 1024 views where fixed-view baselines run out of memory, yielding sustained improvements in novel-view synthesis quality as more observations arrive. The code will be made publicly available upon acceptance.

Problem

• streaming
• causal inference
• novel-view synthesis
• 3D Gaussian Splatting
• online scene reconstruction

Innovation

• Streaming 3D Reconstruction
• Causal Scene Modeling
• Voxel-Aligned Causal Cache
• Feed-Forward 3D Gaussian Splatting
• Memory-Bounded Geometry Representation