Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis

2026-07-24

AI Summary

This work addresses the challenge of visual relocalization in planetary-like environments, where low texture, perceptual aliasing, poor illumination, and sparse viewpoints severely hinder performance. To overcome these limitations, the authors propose a differentiable 3D Gaussian Splatting (3DGS)-based map representation that directly regresses full 6-degree-of-freedom camera poses from a single image, bypassing conventional feature-matching pipelines. The method innovatively integrates multi-view stereo and LiDAR depth data to construct a geometrically accurate 3DGS scene model and introduces a geometry-aware training strategy that jointly optimizes photometric and geometric consistency losses. Experimental results demonstrate that the proposed approach significantly improves both accuracy and robustness on challenging planetary-like relocalization benchmarks.

Abstract

Visual localization becomes extremely challenging in planetary-like terrains characterized by low texture, perceptual aliasing, harsh illumination, and sparse, weakly overlapping viewpoints induced by forward rover motion and unconstrained driving directions. Under these conditions, state-of-the-art image-to-image and image-to-map matching pipelines suffer significant performance degradation. In this work, we propose a visual relocalization method that departs from classical correspondence-based pipelines by directly estimating camera poses against a differentiable map representation built with 3D Gaussian Splatting (3DGS). Our key contribution is a geometry-aware training strategy that combines photometric and geometric losses, where the geometric supervision is provided for the first time by combining multi-view stereo (MVS) and LiDAR depths. We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry, leading to improved photometric and geometric consistency and more robust, accurate single-image 6-DoF pose estimation. Extensive experiments on data acquired in planetary-analog environments validate the effectiveness of our approach, showing substantial gains in relocalization accuracy under challenging conditions. Code is available at https://github.com/DLR-RM/multimodal-gsplat-relocalization.

Problem

• visual relocalization
• low-texture environments
• perceptual aliasing
• sparse views
• planetary-like terrains

Innovation

• 3D Gaussian Splatting
• visual relocalization
• geometry-aware training
• multi-view stereo
• LiDAR fusion