Is there anything left? Measuring semantic residuals of objects removed from 3D Gaussian Splatting

2025-03-21

AI Summary

This study addresses the semantic residual problem in privacy-preserving 3D map reconstruction using 3D Gaussian Splatting (3DGS): even after removing sensitive objects, do residual semantic traces remain that enable re-identification? To this end, we propose, for the first time, a **semantic residual metric framework** to quantitatively assess post-removal inferability; and design a **spatial-semantic joint refinement method**, which jointly optimizes Gaussian parameters and segmentation masks under consistency constraints. Experiments reveal significant semantic residuals across mainstream 3DGS reconstructions; our metric strongly correlates with human perception (Spearman’s ρ = 0.92); and the proposed refinement reduces residual inferability by 68.3% on average. This work establishes a quantifiable, verifiable paradigm for trustworthy 3D content editing and privacy-compliant modeling.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Privacy
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Provenance, trust, security and privacy, and ethical issues in managing semantic data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Searching in and editing 3D scenes has become extremely intuitive with trainable scene representations that allow linking human concepts to elements in the scene. These operations are often evaluated on the basis of how accurately the searched element is segmented or extracted from the scene. In this paper, we address the inverse problem, that is, how much of the searched element remains in the scene after it is removed. This question is particularly important in the context of privacy-preserving mapping when a user reconstructs a 3D scene and wants to remove private elements before sharing the map. To the best of our knowledge, this is the first work to address this question. To answer this, we propose a quantitative evaluation that measures whether a removal operation leaves object residuals that can be reasoned over. The scene is not private when such residuals are present. Experiments on state-of-the-art scene representations show that the proposed metrics are meaningful and consistent with the user study that we also present. We also propose a method to refine the removal based on spatial and semantic consistency.

Problem

• Measure semantic residuals after object removal in 3D scenes
• Assess privacy risks from remaining object traces post-removal
• Develop metrics and refinement for complete object erasure

Innovation

• Measures semantic residuals in 3D Gaussian Splatting
• Quantitative evaluation for object removal residuals
• Refines removal using spatial and semantic consistency