Voyager: Real-Time Splatting City-Scale 3D Gaussians on Your Phone
🤖 AI Summary
To address the challenge of resource-constrained mobile devices and bandwidth bottlenecks hindering real-time rendering of city-scale 3D Gaussian Splatting (3DGS), this paper proposes a cloud-edge collaborative lightweight rendering framework. Our method introduces two key innovations: (1) asynchronous level-of-detail (LOD) search coupled with lookup-table-driven rasterization to significantly reduce GPU computation overhead; and (2) incremental Gaussian streaming guided by dynamic visibility prediction to minimize redundant data transmission. End-to-end optimization achieves over 100× reduction in data transfer volume and up to 8.9× improvement in rendering speed, while preserving high-fidelity visual quality. To the best of our knowledge, this is the first work enabling low-latency, high-fidelity real-time 3DGS rendering at city scale on commodity smartphones.
Application Category
📝 Abstract
3D Gaussian Splatting (3DGS) is an emerging technique for photorealistic 3D scene rendering. However, rendering city-scale 3DGS scenes on mobile devices, e.g., your smartphones, remains a significant challenge due to the limited resources on mobile devices. A natural solution is to offload computation to the cloud; however, naively streaming rendered frames from the cloud to the client introduces high latency and requires bandwidth far beyond the capacity of current wireless networks. In this paper, we propose an effective solution to enable city-scale 3DGS rendering on mobile devices. Our key insight is that, under normal user motion, the number of newly visible Gaussians per second remains roughly constant. Leveraging this, we stream only the necessary Gaussians to the client. Specifically, on the cloud side, we propose asynchronous level-of-detail search to identify the necessary Gaussians for the client. On the client side, we accelerate rendering via a lookup table-based rasterization. Combined with holistic runtime optimizations, our system can deliver low-latency, city-scale 3DGS rendering on mobile devices. Compared to existing solutions, Voyager achieves over 100$ imes$ reduction on data transfer and up to 8.9$ imes$ speedup while retaining comparable rendering quality.
Problem
Research questions and friction points this paper is trying to address.
Enable city-scale 3D Gaussian Splatting on mobile devices
Reduce latency and bandwidth for streaming 3DGS data
Optimize rendering performance on resource-limited smartphones
Innovation
Methods, ideas, or system contributions that make the work stand out.
Stream only necessary Gaussians to client
Asynchronous level-of-detail search on cloud
Lookup table-based rasterization for rendering
🔎 Similar Papers
No similar papers found.
