Have We Scene It All? Scene Graph-Aware Deep Point Cloud Compression

2025-10-09

AI Summary

To address efficient 3D point cloud transmission in bandwidth-constrained and intermittently connected multi-agent systems, this paper proposes the first deep compression framework grounded in semantic scene graphs. Methodologically, it innovatively leverages scene graphs to guide both semantic segmentation and latent-space encoding, incorporating FiLM-based conditional modulation and graph attention mechanisms to jointly model geometric structure and semantic information; at the decoder, a folding-based architecture reconstructs point clouds conditioned on graph node attributes. Evaluated on SemanticKITTI and nuScenes, the framework achieves up to 98% compression ratio while preserving high structural–semantic fidelity. Downstream tasks—including multi-vehicle pose estimation and map fusion—attain performance nearly matching that of raw LiDAR inputs. The core contribution is the first integration of semantic scene graphs into the point cloud compression pipeline, enabling semantic-aware collaborative perception on resource-constrained edge devices.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Learning on the Edge & Model Compression
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Efficient transmission of 3D point cloud data is critical for advanced perception in centralized and decentralized multi-agent robotic systems, especially nowadays with the growing reliance on edge and cloud-based processing. However, the large and complex nature of point clouds creates challenges under bandwidth constraints and intermittent connectivity, often degrading system performance. We propose a deep compression framework based on semantic scene graphs. The method decomposes point clouds into semantically coherent patches and encodes them into compact latent representations with semantic-aware encoders conditioned by Feature-wise Linear Modulation (FiLM). A folding-based decoder, guided by latent features and graph node attributes, enables structurally accurate reconstruction. Experiments on the SemanticKITTI and nuScenes datasets show that the framework achieves state-of-the-art compression rates, reducing data size by up to 98% while preserving both structural and semantic fidelity. In addition, it supports downstream applications such as multi-robot pose graph optimization and map merging, achieving trajectory accuracy and map alignment comparable to those obtained with raw LiDAR scans.

Problem

• Compressing 3D point clouds efficiently under bandwidth constraints
• Preserving structural and semantic fidelity during compression
• Supporting downstream robotics applications with compressed data

Innovation

• Scene graph-based decomposition into semantic patches
• FiLM-conditioned encoders for compact latent representations
• Graph-guided folding decoder for accurate reconstruction