CSGaussian: Progressive Rate-Distortion Compression and Segmentation for 3D Gaussian Splatting

2026-01-19

AI Summary

This work addresses the challenge that existing 3D Gaussian Splatting (3DGS) methods optimize compression and semantic segmentation independently, often compromising transmission efficiency, rendering quality, or downstream task performance. To overcome this limitation, we propose the first joint optimization framework that integrates semantic learning directly into the 3DGS compression pipeline. Our approach employs a lightweight implicit neural hyperprior to enable efficient entropy coding of both color and semantic attributes, circumventing the overhead of explicit mesh representations. Furthermore, we introduce a compression-guided segmentation mechanism that leverages quantization-aware training and quality-aware weighting to enhance feature separability and segmentation robustness. Experiments on the LERF and 3D-OVS datasets demonstrate that our method substantially reduces transmission costs while preserving high-fidelity rendering and achieving strong semantic segmentation performance.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

We present the first unified framework for rate-distortion-optimized compression and segmentation of 3D Gaussian Splatting (3DGS). While 3DGS has proven effective for both real-time rendering and semantic scene understanding, prior works have largely treated these tasks independently, leaving their joint consideration unexplored. Inspired by recent advances in rate-distortion-optimized 3DGS compression, this work integrates semantic learning into the compression pipeline to support decoder-side applications--such as scene editing and manipulation--that extend beyond traditional scene reconstruction and view synthesis. Our scheme features a lightweight implicit neural representation-based hyperprior, enabling efficient entropy coding of both color and semantic attributes while avoiding costly grid-based hyperprior as seen in many prior works. To facilitate compression and segmentation, we further develop compression-guided segmentation learning, consisting of quantization-aware training to enhance feature separability and a quality-aware weighting mechanism to suppress unreliable Gaussian primitives. Extensive experiments on the LERF and 3D-OVS datasets demonstrate that our approach significantly reduces transmission cost while preserving high rendering quality and strong segmentation performance.

Problem

• 3D Gaussian Splatting
• rate-distortion optimization
• compression
• segmentation
• semantic scene understanding

Innovation

• 3D Gaussian Splatting
• rate-distortion optimization
• semantic segmentation
• hyperprior
• quantization-aware training