Adaptive Anchor Policies for Efficient 4D Gaussian Streaming

2026-03-17

AI Summary

This work addresses the limitations of existing 4D Gaussian streaming reconstruction methods, which rely on fixed anchor sampling strategies—such as farthest point sampling (FPS)—that fail to adapt to scene complexity and often result in inefficient resource allocation or degraded reconstruction quality. To overcome this, we propose Efficient Gaussian Streaming (EGS), the first approach to integrate reinforcement learning into the anchor selection process, enabling budget-aware adaptive sampling. EGS jointly optimizes the number, spatial positions, and associated information of anchors while preserving the original reconstruction backbone. Experiments demonstrate that EGS achieves a PSNR improvement of 0.52–0.61 dB using only 256 anchors—32× fewer than the conventional 8192—and accelerates reconstruction by 1.29–1.35×. Under high-quality fine-tuning, EGS matches the performance of full-anchor baselines at a substantially reduced computational budget.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds
• Computer Vision: 3D Computer Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Dynamic scene reconstruction with Gaussian Splatting has enabled efficient streaming for real-time rendering and free-viewpoint video. However, most pipelines rely on fixed anchor selection such as Farthest Point Sampling (FPS), typically using 8,192 anchors regardless of scene complexity, which over-allocates computation under strict budgets. We propose Efficient Gaussian Streaming (EGS), a plug-in, budget-aware anchor sampler that replaces FPS with a reinforcement-learned policy while keeping the Gaussian streaming reconstruction backbone unchanged. The policy jointly selects an anchor budget and a subset of informative anchors under discrete constraints, balancing reconstruction quality and runtime using spatial features of the Gaussian representation. We evaluate EGS in two settings: fast rendering, which prioritizes runtime efficiency, and high-quality refinement, which enables additional optimization. Experiments on dynamic multi-view datasets show consistent improvements in the quality--efficiency trade-off over FPS sampling. On unseen data, in fast rendering at 256 anchors ($32\times$ fewer than 8,192), EGS improves PSNR by $+0.52$--$0.61$\,dB while running $1.29$--$1.35\times$ faster than IGS@8192 (N3DV and MeetingRoom). In high-quality refinement, EGS remains competitive with the full-anchor baseline at substantially lower anchor budgets. \emph{Code and pretrained checkpoints will be released upon acceptance.} \keywords{4D Gaussian Splatting \and 4D Gaussian Streaming \and Reinforcement Learning}

Problem

• 4D Gaussian Splatting
• 4D Gaussian Streaming
• Anchor Selection
• Efficiency-Quality Trade-off
• Dynamic Scene Reconstruction

Innovation

• 4D Gaussian Splatting
• Adaptive Anchor Sampling
• Reinforcement Learning
• Efficient Streaming
• Dynamic Scene Reconstruction