Beyond Darkness: Thermal-Supervised 3D Gaussian Splatting for Low-Light Novel View Synthesis

2025-11-17

AI Summary

To address geometric distortion, chromatic inconsistency, and radiometric instability in novel view synthesis (NVS) under extremely low-light conditions, this paper proposes DTGS—a thermal-aware joint optimization framework. Methodologically, DTGS innovatively integrates a Retinex-inspired illumination decomposition module into the 3D Gaussian Splatting (3DGS) optimization pipeline, establishing an RGB–thermal (T) dual-modal collaborative reconstruction paradigm. A thermal-supervised branch is introduced to decouple geometry and illumination estimation, enabling joint enhancement and reconstruction optimization. Furthermore, we construct RGBT-LOW—the first multimodal dataset specifically designed for low-light NVS. Extensive experiments on RGBT-LOW demonstrate that DTGS significantly improves radiometric consistency, geometric accuracy, and cross-view color stability, outperforming state-of-the-art methods.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Under extremely low-light conditions, novel view synthesis (NVS) faces severe degradation in terms of geometry, color consistency, and radiometric stability. Standard 3D Gaussian Splatting (3DGS) pipelines fail when applied directly to underexposed inputs, as independent enhancement across views causes illumination inconsistencies and geometric distortion. To address this, we present DTGS, a unified framework that tightly couples Retinex-inspired illumination decomposition with thermal-guided 3D Gaussian Splatting for illumination-invariant reconstruction. Unlike prior approaches that treat enhancement as a pre-processing step, DTGS performs joint optimization across enhancement, geometry, and thermal supervision through a cyclic enhancement-reconstruction mechanism. A thermal supervisory branch stabilizes both color restoration and geometry learning by dynamically balancing enhancement, structural, and thermal losses. Moreover, a Retinex-based decomposition module embedded within the 3DGS loop provides physically interpretable reflectance-illumination separation, ensuring consistent color and texture across viewpoints. To evaluate our method, we construct RGBT-LOW, a new multi-view low-light thermal dataset capturing severe illumination degradation. Extensive experiments show that DTGS significantly outperforms existing low-light enhancement and 3D reconstruction baselines, achieving superior radiometric consistency, geometric fidelity, and color stability under extreme illumination.

Problem

• Addresses novel view synthesis degradation under extreme low-light conditions
• Solves illumination inconsistencies in 3D Gaussian Splatting with thermal guidance
• Ensures consistent color and geometry through joint enhancement-reconstruction optimization

Innovation

• Thermal-guided 3D Gaussian Splatting for illumination-invariant reconstruction
• Joint optimization of enhancement, geometry, and thermal supervision
• Retinex-based decomposition module embedded within 3DGS loop