Path-Traced Inverse Rendering with Global Illumination in 3D Gaussian Fields

2026-06-08

AI Summary

Existing inverse rendering methods rely on rasterization pipelines and simplified lighting models that neglect indirect illumination, leading to inconsistencies in material and lighting estimation under path tracing and resulting in visual artifacts. This work proposes the first splatting-free inverse rendering framework based on path tracing, which jointly optimizes forward light transport and backward gradient propagation within a unified ray tracing pipeline, enabling accurate reconstruction of materials and environment lighting under multi-bounce global illumination. By formulating a path-space equivalent interaction model within a 3D Gaussian field, the method achieves unbiased path tracing and path-space gradient replay, eliminating dependence on screen-space buffers. The approach significantly improves material inversion accuracy and produces more realistic shadows, reflections, and relighting effects.

Abstract

Ray tracing enables 3D Gaussian fields to serve as a representation for physically based light transport. Faithful inverse rendering requires forward rendering and backward optimization to be defined within a consistent light-transport pipeline. Existing inverse rendering methods estimate G-buffers via splatting and optimize materials in screen space, tying the recovered properties to a rasterization-based pipeline. This pipeline mismatch, together with simplified rendering equations that neglect indirect illumination, often leads to inconsistent shading, visible artifacts, and inaccurate material-lighting estimation under path-traced rendering. Therefore, we propose a splatting-free path-traced inverse rendering framework for 3D Gaussian fields, where forward light transport and backward gradient propagation are defined within a unified ray-tracing pipeline. Our key idea is to define a path-space equivalent interaction model for overlapping Gaussian primitives, under which Monte-Carlo-based path tracing is unbiased for the induced light-transport integral, while pathwise gradients are replayed over the same ray-traced interactions rather than splatting-derived screen-space buffers. The framework optimizes materials and a compact Spherical-Gaussian environment under the full rendering equation with ray-traced visibility and multi-bounce light transport. Extensive experiments demonstrate competitive material inversion and improved path-traced rendering quality, producing more plausible shadows, reflections, and relighting results under global illumination.

Problem

• inverse rendering
• global illumination
• path tracing
• 3D Gaussian fields
• light transport

Innovation

• Path-Traced Inverse Rendering
• 3D Gaussian Fields
• Global Illumination
• Ray Tracing
• Differentiable Rendering