RRG-SLAM: Real-time Reflection-aware Gaussian SLAM for Indoor Scenes

2026-09-28

AI Summary

This study addresses the challenges of localization drift and reconstruction distortion in SLAM caused by indoor specular reflections. We propose the first real-time reflection-aware Gaussian SLAM system. Methodologically, we introduce a novel hybrid TSDF-3DGS representation that integrates multi-cue plane detection with reflection-aware tracking to explicitly decouple the diffuse base scene from specular reflections. Furthermore, a three-pass rendering pipeline coupled with an online optimization and pruning strategy is designed to achieve efficient appearance separation. The proposed system significantly outperforms existing methods in 3D reconstruction accuracy, camera tracking robustness, and novel view synthesis quality, while maintaining real-time performance.

Abstract

We introduce the first real-time reflection-aware Gaussian SLAM system for indoor scenes. The system features a reflection-aware TSDF-Gaussian hybrid representation that explicitly separates diffuse scene appearance from reflection components. The base scene is modeled by a TSDF volume and a set of base Gaussians capturing geometry and diffuse appearance, while planar reflections are represented by reflection Gaussian groups associated with detected reflective planes. The rendering is performed in three passes: TSDF raycasting first yields surface color, depth, plane IDs and reflection masks; base Gaussians are then rendered order-independently with depth culling and combined with the TSDF output to form the base image; finally, under the guidance of the plane ID map, reflection Gaussians from different reflection groups are rasterized only into their corresponding planar regions to generate the reflection image, which is subsequently composited with the base image via the reflection mask to produce the final output. For online reconstruction, our system first estimates the camera pose through reflection-aware tracking to suppress interference of reflection-dominated regions. It then identifies reflective planes using geometric, semantic, and temporal cues, and fuses the observations into the augmented TSDF volume with reflection-aware attributes. Afterwards the base and reflection Gaussians are initialized, optimized, and pruned online to maintain both reconstruction quality and efficiency. Experiments on a variety of datasets show that our method outperforms existing SLAM systems in reconstruction quality, tracking robustness, and novel-view rendering for indoor environments with reflections, while preserving real-time performance.

Problem

• Reflection-aware SLAM
• Indoor scene reconstruction
• Real-time tracking
• Gaussian SLAM
• Novel-view rendering

Innovation

• Reflection-aware SLAM
• 3D Gaussian Splatting
• TSDF-Gaussian Hybrid Representation
• Indoor Scene Reconstruction
• Real-time Rendering