Casual Flash Lighting for Gaussian Splat Inverse Rendering

2026-10-05

AI Summary

This study addresses the inherent ambiguity in recovering geometry, materials, and illumination from static lighting alone by proposing an active flash-assisted inverse rendering method that requires neither darkrooms nor specialized hardware. By combining casually captured indoor flash and no-flash images, the approach leverages flash residuals to constrain albedo and employs 2D Gaussian Splatting (2DGS) for view-consistent material decomposition. A core innovation is the introduction of a 2DGS-anchored diffuse field, which binds hash-encoded MLP queries to rasterized depth, effectively eliminating cross-view alpha-blending drift errors. Experimental results demonstrate that the proposed method outperforms six baselines across multiple scenes, achieving a 4.17 dB improvement in relighting PSNR and significantly enhancing material parameter estimation accuracy.

Abstract

Recovering geometry, materials, and lighting from photographs is highly ambiguous when only static illumination is available. Active-lighting setups reduce the ambiguity but require dark rooms or specialized hardware. Instead, we synergize both static and flash lighting from casual indoor capture, with the flash on or off, each from independent viewpoints. The flash residual constrains albedo and the BRDF, while static lighting captures grazing-angle specular highlights that flash misses. With a 2DGS reconstruction framing, our key contribution is a GS-anchored diffuse field: a hash-encoded MLP is queried at the rasterized 2DGS depth. As it depends only on world position, it is view consistent in 3D and allows the flash residual to drive material decomposition instead of being absorbed by alpha-blending drift across views. At the same time, we render static lighting with deferred shading such that it can also supervise material decomposition. On five synthetic and three real indoor scenes, our method outperforms six recent baselines on diffuse color, albedo and roughness material parameters, and in relighting where PSNR improves by 4.17 dB over the next-best baseline.

Problem

• Inverse Rendering
• Gaussian Splatting
• Material Decomposition
• Flash Lighting
• Relighting

Innovation

• Inverse Rendering
• 2D Gaussian Splatting
• Flash Lighting
• Material Decomposition
• Deferred Shading