ROSGS: Relightable Outdoor Scenes With Gaussian Splatting

2025-09-14

AI Summary

Outdoor scenes pose significant challenges for geometry-albedo-illumination disentangled reconstruction due to unbounded extents and highly variable lighting. Existing NeRF- and 3D Gaussian Splatting (3DGS)-based approaches suffer from prohibitive computational costs and inadequate modeling of high-frequency illumination, resulting in limited relighting accuracy and rendering efficiency. To address these issues, we propose a two-stage relightable reconstruction framework. In the first stage, we perform efficient monocular normal-prior-guided 2D Gaussian splatting for geometric reconstruction. In the second stage, we introduce a hybrid illumination model that jointly employs spherical Gaussians—capturing high-frequency direct sunlight—and spherical harmonic coefficients—learning low-frequency skylight and radiative transport—for illumination disentanglement. Our method achieves high-fidelity geometry reconstruction while significantly improving relighting accuracy and enabling real-time rendering. Extensive evaluations on multiple outdoor datasets demonstrate state-of-the-art quantitative metrics and visual quality.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Feature Construction/Reformulation

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Ad search and search for Web retail

Abstract

Image data captured outdoors often exhibit unbounded scenes and unconstrained, varying lighting conditions, making it challenging to decompose them into geometry, reflectance, and illumination. Recent works have focused on achieving this decomposition using Neural Radiance Fields (NeRF) or the 3D Gaussian Splatting (3DGS) representation but remain hindered by two key limitations: the high computational overhead associated with neural networks of NeRF and the use of low-frequency lighting representations, which often result in inefficient rendering and suboptimal relighting accuracy. We propose ROSGS, a two-stage pipeline designed to efficiently reconstruct relightable outdoor scenes using the Gaussian Splatting representation. By leveraging monocular normal priors, ROSGS first reconstructs the scene's geometry with the compact 2D Gaussian Splatting (2DGS) representation, providing an efficient and accurate geometric foundation. Building upon this reconstructed geometry, ROSGS then decomposes the scene's texture and lighting through a hybrid lighting model. This model effectively represents typical outdoor lighting by employing a spherical Gaussian function to capture the directional, high-frequency components of sunlight, while learning a radiance transfer function via Spherical Harmonic coefficients to model the remaining low-frequency skylight comprehensively. Both quantitative metrics and qualitative comparisons demonstrate that ROSGS achieves state-of-the-art performance in relighting outdoor scenes and highlight its ability to deliver superior relighting accuracy and rendering efficiency.

Problem

• Decomposing outdoor scenes into geometry, reflectance, and illumination
• Overcoming computational overhead and low-frequency lighting limitations
• Achieving efficient reconstruction with accurate relighting for unbounded scenes

Innovation

• Two-stage Gaussian Splatting pipeline
• Monocular normal priors for geometry
• Hybrid spherical Gaussian lighting model