PePESeg3D: Perception Prior Enhances Multi-Scale Segmentation for 3D Gaussian Splatting

2026-09-23

AI Summary

This study addresses the disconnection between geometry and semantics in multi-scale 3D Gaussian Splatting segmentation, as well as the reliance of feature learning on incomplete mask supervision. To overcome these limitations, this work proposes the Perceptual Prior Injection framework (PePE). This method pioneers the synergistic application of monocular depth and mask constraints to both upstream geometric reconstruction and downstream contrastive feature learning, enabling semantically consistent multi-scale 3D scene segmentation. By integrating dense depth-color cues with view-consistent centroid supervision, the proposed approach achieves state-of-the-art performance on benchmarks such as SPIn-NeRF across both multi-scale segmentation and scene reconstruction tasks.

Abstract

Recent advancements in 3D Gaussian Splatting (3DGS) have extended its capabilities to multi-scale segmentation. Existing methods reconstruct a scene with Gaussian primitives and learn multi-scale segmentation features separately, which leaves the geometry unaware of semantic structure and the feature learning dependent on incomplete mask supervision. To address these limitations, we present PePESeg3D, a novel framework that injects perception priors into a multi-scale 3D Gaussian segmentation pipeline. To fully exploit perception priors, we integrate them not only into contrastive feature learning but also into the upstream geometry reconstruction. Specifically, PePE Reconstruction incorporates monocular depth and mask constraints to ensure semantically coherent object structures. Building on this aligned geometry, PePE Contrastive Learning leverages dense depth-color cues and view-consistent centroid supervision to compensate for the incompleteness of multi-scale masks obtained from a 2D foundation model. Extensive experiments on the SPIn-NeRF, LERF-Mask, and NVOS benchmarks demonstrate that PePESeg3D achieves state-of-the-art performance in both multi-scale segmentation and scene reconstruction, highlighting the importance of integrating perception priors into both geometry optimization and feature learning for accurate multi-scale 3D segmentation. Our code is available at https://github.com/BeCow5X5/PePESeg3D.

Problem

• 3D Gaussian Splatting
• Multi-scale Segmentation
• Perception Prior
• Geometry Reconstruction
• Feature Learning

Innovation

• 3D Gaussian Splatting
• Perception Priors
• Multi-Scale Segmentation
• Contrastive Learning
• Geometry Reconstruction