PointSplat: Compact Gaussian Splatting via Human-Centric Prediction

2026-06-30

AI Summary

Existing immersive live-streaming systems redundantly encode human content across multiple views, leading to inefficient representations that struggle to balance high fidelity with compactness. This work proposes a human-centric 3D Gaussian splatting method that directly predicts Gaussian primitives in 3D space from input point clouds, thereby eliminating multi-view redundancy. By leveraging geometric proxies, ray culling, and a point-image Transformer to fuse appearance and geometry features, the approach generates Gaussians exclusively in foreground regions, significantly enhancing both compactness and rendering quality. Evaluated across multiple datasets, the method achieves superior rendering efficiency and robustness, remains insensitive to variations in view count and resolution, and substantially reduces the total number of Gaussian primitives required.

Abstract

Producing 3D human representations from input views on the fly is essential for immersive live streaming systems, where representation compactness is as critical as high fidelity given limited computational power and transmission bandwidth. Although recent feed-forward reconstruction methods achieve impressive quality through the view-centric prediction of 3D representations, they repeatedly encode the same subject content across multiple views, leading to significant inter-view redundancy. Our key insight is to perform predictions directly in 3D space, enabling the network to learn and produce a highly compact representation. To this end, we propose PointSplat, a novel human-centric approach that directly infers Gaussian primitives from an input point set. The proposed method first estimates a coarse geometric proxy and performs ray casting to prune redundant points and establish explicit 2D--3D correspondences. Subsequently, it employs a Point-Image Transformer to fuse appearance and geometry features, predicting Gaussian attributes in a single forward pass. This design restricts predictions to foreground regions of interest, substantially reducing the total number of Gaussians while improving novel-view rendering quality. Extensive experiments demonstrate that PointSplat achieves higher efficiency and quality while exhibiting strong robustness to variations in view count and image resolution across multiple datasets.

Problem

• 3D human representation
• compactness
• view redundancy
• real-time streaming
• Gaussian splatting

Innovation

• Gaussian Splatting
• Human-Centric Prediction
• 3D Reconstruction
• Point-Image Transformer
• Compact Representation