TASE: Truncation-Aware Semantic Embeddings for 3D Scene Understanding and Editing

2026-06-02

AI Summary

This work addresses the challenge of achieving both high controllability and substantial geometric modifications in high-fidelity semantic 3D scene editing. The authors propose a truncation-aware embedding space that projects pretrained 2D semantic features into a structured latent representation, enabling explicit control ranging from abstract to fine-grained edits through adjustable feature channel counts. To enhance edit quality, they introduce scale- and translation-equivariant losses, integrate multi-view consistency optimization, and fine-tune diffusion models within this framework. The resulting method supports large-scale geometric transformations while effectively suppressing artifacts, significantly outperforming existing approaches in text-driven 3D scene editing tasks.

Abstract

High-fidelity semantic 3D scene representations are crucial for numerous applications, including robotics, autonomous driving, and simulation. Beyond this, the ability to edit such representations enables developers to adapt these applications more easily to specific target scenarios. Current approaches provide limited support for controllable editing. We introduce TASE, a method that projects pretrained 2D semantic features into a truncation-aware embedding space to enable flexible 3D scene editing. Our method explicitly optimizes a feature space in which progressively reducing feature channels yields increasingly abstract semantic representations, while retaining more channels preserves fine-grained detail. Additionally, we improve multi-view consistency of the features using a scale- and translation-equivariance loss. The resulting truncation-aware embedding space enables text-driven edits to 3D scenes, providing explicit control over how strongly edits adhere to the original scene content and allowing more substantial modifications than prior methods. Moreover, we propose a finetuning stage for the editing diffusion model to mitigate artifacts caused by geometric changes. Experimental results demonstrate competitive performance in 3D scene editing, substantially outperforming prior methods on edits involving large geometric modifications.

Problem

• 3D scene editing
• semantic embeddings
• truncation-aware
• controllable editing
• geometric modifications

Innovation

• truncation-aware embedding
• 3D scene editing
• semantic feature projection
• equivariance loss
• text-driven editing