GaRe: Relightable 3D Gaussian Splatting for Outdoor Scenes from Unconstrained Photo Collections

2025-07-28

AI Summary

This work addresses relightable 3D Gaussian splatting for unconstrained outdoor photo collections. We propose the first framework enabling physically consistent joint decomposition and editing of direct sunlight, sky radiance, and indirect illumination. Methodologically, we introduce an intrinsic-image-decomposition-driven 3D Gaussian lattice representation, coupled with a residual solar visibility extraction module, region-wise structural consistency loss, and ray-traced shadow simulation to achieve disentangled, interpretable lighting modeling and dynamic shadow synthesis. Compared to prior approaches, our method retains state-of-the-art fidelity in novel-view synthesis while significantly improving shadow controllability, naturalness, and diversity. It enables photorealistic relighting and interactive light-shadow editing—marking a substantive advance in editable, physics-aware 3D scene reconstruction from unstructured outdoor imagery.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We propose a 3D Gaussian splatting-based framework for outdoor relighting that leverages intrinsic image decomposition to precisely integrate sunlight, sky radiance, and indirect lighting from unconstrained photo collections. Unlike prior methods that compress the per-image global illumination into a single latent vector, our approach enables simultaneously diverse shading manipulation and the generation of dynamic shadow effects. This is achieved through three key innovations: (1) a residual-based sun visibility extraction method to accurately separate direct sunlight effects, (2) a region-based supervision framework with a structural consistency loss for physically interpretable and coherent illumination decomposition, and (3) a ray-tracing-based technique for realistic shadow simulation. Extensive experiments demonstrate that our framework synthesizes novel views with competitive fidelity against state-of-the-art relighting solutions and produces more natural and multifaceted illumination and shadow effects.

Problem

• Outdoor scene relighting from unconstrained photos
• Accurate sunlight and shadow effect generation
• Physically interpretable illumination decomposition

Innovation

• Residual-based sun visibility extraction method
• Region-based supervision with structural loss
• Ray-tracing-based realistic shadow simulation