ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion

2026-07-22

AI Summary

This work addresses the limitations of existing feed-forward 3D Gaussian splatting methods, whose primitive counts and distributions are constrained by image resolution and viewpoint due to pixel alignment, leading to redundant representations that fail to adapt to scene complexity. The authors propose ATSplat, a novel framework that introduces learnable sparse 3D tokens as a scene skeleton within a feed-forward paradigm. By leveraging rendering-error-driven uncertainty scores, the method dynamically expands tokens in high-uncertainty regions, enabling adaptive concentration of Gaussian primitives where scene detail is most complex. This approach decouples primitive layout from the input image grid, substantially improving representation efficiency. On RealEstate10K and DL3DV, ATSplat achieves state-of-the-art rendering quality with only 311k Gaussians—5.7× fewer than dense feed-forward counterparts—while enabling sub-second reconstruction on a single GPU and novel-view synthesis at 1136 FPS (512×960).

Abstract

3D Gaussian Splatting (3DGS) achieves high-quality novel-view synthesis by optimizing freely placed primitives in 3D and adaptively densifying them in under-reconstructed regions. However, this scene-adaptive capacity allocation is largely lost in existing feed-forward 3DGS methods, which commonly regress Gaussians at input pixels and lift them along camera rays. Such pixel-aligned formulations make the number and placement of primitives depend on image resolution and input viewpoints rather than scene complexity, resulting in dense and often redundant Gaussian sets. We present ATSplat, a feed-forward 3DGS framework that restores the adaptive allocation capability of 3DGS optimization through Adaptive 3D Tokens. ATSplat first lifts coarse patch-level depth and camera cues into sparse 3D anchor tokens, forming a compact scaffold of the scene. Each token is then regressed into local Gaussians with learnable 3D offsets, decoupling primitive placement from input image grids. An Adaptive Token Expansion module predicts a token-level uncertainty score, supervised by rendering error maps, and selectively expands high-uncertainty tokens through learnable expansion layers. This sparse-to-adaptive formulation enables ATSplat to concentrate primitives in challenging regions while maintaining a compact representation. Experiments on two representative datasets, RealEstate10K and DL3DV, show that ATSplat achieves state-of-the-art rendering quality while reducing the number of Gaussians by more than $5.7\times$ compared with dense feed-forward 3DGS methods. From 12 input images at $512 \times 960$ resolution, ATSplat completes reconstruction in less than a second using a single commercial GPU, and renders high-quality novel views at 1136 FPS ($512 \times 960$) with only 311K Gaussians.

Problem

• 3D Gaussian Splatting
• adaptive allocation
• feed-forward reconstruction
• primitive redundancy
• scene complexity

Innovation

• Adaptive Token Expansion
• Feed-forward 3D Gaussian Splatting
• Sparse 3D Representation
• Scene-adaptive Allocation
• Efficient Novel-view Synthesis