TranSplat: Lighting-Consistent Cross-Scene Object Transfer with 3D Gaussian Splatting

2025-03-28

AI Summary

This work addresses two key challenges in cross-scene 3D object transfer: precise cross-domain object extraction and material-free, illumination-consistent relighting. We propose a spherical harmonic–based Gaussian radiance transfer function that enables physics-aware relighting without explicit BRDF estimation—marking the first such approach. Integrated with mask-driven fine-grained 3D segmentation and user-guided end-to-end pose optimization, our method leverages 3D Gaussian splatting and differentiable rendering. Evaluated on both synthetic and real-world datasets, it significantly outperforms prior methods in fidelity of object extraction, naturalness of illumination adaptation, and seamlessness of scene embedding—yielding photorealistic visual results. The core contributions are (1) a material-free spherical harmonic radiance transfer mechanism enabling physically grounded relighting, and (2) a novel co-optimization paradigm jointly refining segmentation masks and object pose.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Security and Privacy: Data transparency and provenance

Abstract

We present TranSplat, a 3D scene rendering algorithm that enables realistic cross-scene object transfer (from a source to a target scene) based on the Gaussian Splatting framework. Our approach addresses two critical challenges: (1) precise 3D object extraction from the source scene, and (2) faithful relighting of the transferred object in the target scene without explicit material property estimation. TranSplat fits a splatting model to the source scene, using 2D object masks to drive fine-grained 3D segmentation. Following user-guided insertion of the object into the target scene, along with automatic refinement of position and orientation, TranSplat derives per-Gaussian radiance transfer functions via spherical harmonic analysis to adapt the object's appearance to match the target scene's lighting environment. This relighting strategy does not require explicitly estimating physical scene properties such as BRDFs. Evaluated on several synthetic and real-world scenes and objects, TranSplat yields excellent 3D object extractions and relighting performance compared to recent baseline methods and visually convincing cross-scene object transfers. We conclude by discussing the limitations of the approach.

Problem

• Realistic cross-scene object transfer using 3D Gaussian Splatting
• Precise 3D object extraction without material property estimation
• Faithful relighting adaptation to target scene's lighting environment

Innovation

• 3D Gaussian Splatting for object transfer
• 2D masks drive fine-grained 3D segmentation
• Spherical harmonics for lighting adaptation