TR-Gaussians: High-fidelity Real-time Rendering of Planar Transmission and Reflection with 3D Gaussian Splatting

2025-11-17

AI Summary

This paper addresses the challenge of high-fidelity real-time rendering of planar transmissive and reflective surfaces (e.g., glass) in indoor scenes. We propose the first 3D Gaussian Splatting method explicitly designed for joint transmission-reflection modeling. Our key contributions are: (1) introducing learnable reflection planes and mirrored Gaussians to explicitly encode view-dependent reflection and transmission; (2) designing a differentiable, Fresnel-based compositing strategy for physically inspired light-path synthesis; and (3) establishing a multi-stage optimization framework incorporating color, geometry, and opacity perturbation constraints. Evaluated on multiple indoor datasets, our method generates high-quality novel-view images at real-time frame rates. Quantitatively, it surpasses state-of-the-art methods across PSNR, SSIM, and LPIPS metrics; qualitatively, it delivers superior visual fidelity in handling complex glass interactions.

Technology Category

• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

We propose Transmission-Reflection Gaussians (TR-Gaussians), a novel 3D-Gaussian-based representation for high-fidelity rendering of planar transmission and reflection, which are ubiquitous in indoor scenes. Our method combines 3D Gaussians with learnable reflection planes that explicitly model the glass planes with view-dependent reflectance strengths. Real scenes and transmission components are modeled by 3D Gaussians and the reflection components are modeled by the mirrored Gaussians with respect to the reflection plane. The transmission and reflection components are blended according to a Fresnel-based, view-dependent weighting scheme, allowing for faithful synthesis of complex appearance effects under varying viewpoints. To effectively optimize TR-Gaussians, we develop a multi-stage optimization framework incorporating color and geometry constraints and an opacity perturbation mechanism. Experiments on different datasets demonstrate that TR-Gaussians achieve real-time, high-fidelity novel view synthesis in scenes with planar transmission and reflection, and outperform state-of-the-art approaches both quantitatively and qualitatively.

Problem

• Modeling planar transmission and reflection effects in indoor scenes
• Achieving real-time high-fidelity rendering with 3D Gaussian representation
• Optimizing complex appearance effects under varying viewpoints

Innovation

• Combines 3D Gaussians with learnable reflection planes
• Models reflections using mirrored Gaussians with Fresnel blending
• Uses multi-stage optimization with opacity perturbation mechanism