GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completion

2026-03-22

AI Summary

This work addresses the challenges of insufficient voxel-image feature alignment and limited geometric expressiveness in monocular image-driven 3D semantic scene completion by proposing a two-stage, grid-native framework. The core innovations include a Gaussian anchoring mechanism to enhance cross-modal alignment and a directional, anisotropic Gaussian-triplane refinement module that effectively models surface tangency, scale variation, and occlusion asymmetry. By integrating Gaussian-weighted aggregation of FPN features with a local-global cooperative triplane alignment strategy, the method achieves seamless fusion of Gaussian characteristics into voxel representations. On SemanticKITTI, the first stage improves occupancy prediction by +1.0% Recall, +2.0% Precision, and +1.8% IoU, while the second stage boosts semantic prediction by +1.8% IoU and +0.8% mIoU.

Technology Category

• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

We present \emph{GaussianSSC}, a two-stage, grid-native and triplane-guided approach to semantic scene completion (SSC) that injects the benefits of Gaussians without replacing the voxel grid or maintaining a separate Gaussian set. We introduce \emph{Gaussian Anchoring}, a sub-pixel, Gaussian-weighted image aggregation over fused FPN features that tightens voxel--image alignment and improves monocular occupancy estimation. We further convert point-like voxel features into a learned per-voxel Gaussian field and refine triplane features via a triplane-aligned \emph{Gaussian--Triplane Refinement} module that combines \emph{local gathering} (target-centric) and \emph{global aggregation} (source-centric). This directional, anisotropic support captures surface tangency, scale, and occlusion-aware asymmetry while preserving the efficiency of triplane representations. On SemanticKITTI~\cite{behley2019semantickitti}, GaussianSSC improves Stage~1 occupancy by +1.0\% Recall, +2.0\% Precision, and +1.8\% IoU over state-of-the-art baselines, and improves Stage~2 semantic prediction by +1.8\% IoU and +0.8\% mIoU.

Problem

• Semantic Scene Completion
• 3D Reconstruction
• Monocular Occupancy Estimation
• Semantic Segmentation
• Voxel-based Representation

Innovation

• Gaussian Anchoring
• Triplane-Guided Refinement
• Directional Gaussian Fields
• Semantic Scene Completion
• Anisotropic Feature Aggregation