Federated 3D Gaussian Splatting for Large-Scale Scene Reconstruction at Wireless Edge

2026-09-26

AI Summary

This study addresses the challenges of resource constraints, model heterogeneity, and privacy leakage in large-scale scene reconstruction over wireless edges. To this end, it proposes a resource-efficient federated learning framework tailored for 3D Gaussian Splatting. Methodologically, the framework introduces an edge-side adaptive pruning and parameter-free recovery mechanism based on an importance-to-delay ratio, alongside a camera-parameter-free global aggregation strategy that enables collaborative training of heterogeneous models while preserving privacy. Experimental results demonstrate that the proposed approach significantly accelerates convergence and reduces communication latency. By maintaining high rendering quality, the framework comprehensively outperforms existing baselines, offering an effective solution for privacy-preserving, large-scale 3D reconstruction in resource-constrained wireless edge environments.

Abstract

Three-dimensional (3D) Gaussian splatting (3D-GS) has emerged as a promising technique for large-scale scene reconstruction due to its high rendering efficiency and fidelity. However, the training of large-scale 3D-GS models at wireless edge faces various technical challenges including the limited communication, computation, and graphics processing unit (GPU) memory resources at edge devices, the structural inconsistency issue across local models hindering their effective aggregation, as well as privacy leakage risks associated with raw visual content and camera parameters. To address these challenges, this paper proposes a novel resource-efficient federated learning framework for efficiently training 3D-GS models of large scenes under severe resource constraints. First, we propose an on-device model lightweighting mechanism that adaptively selects and prunes Gaussian points to balance the rendering quality and training efficiency. In this mechanism, we quantitatively evaluate the importance of different Gaussian points at each device to facilitate the pruning, and use a novel importance-to-latency ratio criterion to determine the number of pruned Gaussian points under GPU memory and computation/communication latency constraints. Furthermore, we develop a 3D-GS model recovery mechanism that restores structural consistency across local 3D-GS models without accessing private camera parameters, enabling their effective aggregation towards a global model. Finally, extensive experiments show that our approach significantly accelerates convergence, maintains high rendering quality, and reduces training latency compared to state-of-the-art federated 3D-GS baselines.

Problem

• Federated Learning
• 3D Gaussian Splatting
• Edge Computing
• Scene Reconstruction
• Privacy Preservation

Innovation

• Federated Learning
• 3D Gaussian Splatting
• Model Lightweighting
• Structure Consistency Recovery
• Edge Computing