TransSplat: Unbalanced Semantic Transport for Language-Driven 3DGS Editing

2026-04-21

AI Summary

Existing language-driven 3D Gaussian Splatting (3DGS) editing methods lack explicit modeling of the semantic correspondence between 2D editing evidence and 3D Gaussians, leading to poor local accuracy and structural inconsistency. This work formulates the problem as a multi-view, imbalanced semantic transfer task, establishing correspondences between visible Gaussians and view-specific editing prototypes to recover a canonical 3D editing field shared across views, thereby unifying appearance updates. A transfer residual is introduced to suppress erroneous edits in non-target regions. By explicitly linking 2D edits with 3D semantics through a semantic transfer mechanism—introduced here for the first time in 3DGS editing—the method significantly enhances local control precision and mitigates edit leakage. Experiments demonstrate that our approach outperforms existing view-consistency-based methods in both local accuracy and structural consistency.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Language-driven 3D Gaussian Splatting (3DGS) editing provides a more convenient approach for modifying complex scenes in VR/AR. Standard pipelines typically adopt a two-stage strategy: first editing multiple 2D views, and then optimizing the 3D representation to match these edited observations. Existing methods mainly improve view consistency through multi-view feature fusion, attention filtering, or iterative recalibration. However, they fail to explicitly address a more fundamental issue: the semantic correspondence between edited 2D evidence and 3D Gaussians. To tackle this problem, we propose TransSplat, which formulates language-driven 3DGS editing as a multi-view unbalanced semantic transport problem. Specifically, our method establishes correspondences between visible Gaussians and view-specific editing prototypes, thereby explicitly characterizing the semantic relationship between edited 2D evidence and 3D Gaussians. It further recovers a cross-view shared canonical 3D edit field to guide unified 3D appearance updates. In addition, we use transport residuals to suppress erroneous edits in non-target regions, mitigating edit leakage and improving local control precision. Qualitative and quantitative results show that, compared with existing 3D editing methods centered on enhancing view consistency, TransSplat achieves superior performance in local editing accuracy and structural consistency.

Problem

• language-driven editing
• 3D Gaussian Splatting
• semantic correspondence
• multi-view editing
• 3D scene editing

Innovation

• semantic transport
• language-driven editing
• 3D Gaussian Splatting
• view consistency
• canonical edit field