🤖 AI Summary
Existing camera pose estimation methods suffer from low initial pose accuracy, reliance on task-specific descriptors, or the need for network fine-tuning or scene reconstruction. To address these limitations, we propose GS-SMC—the first lightweight pose optimization framework based on 3D Gaussian Splatting (3DGS). GS-SMC leverages a pre-trained 3DGS model to render multi-view images, establishes 2D–2D correspondences via feature matching, and iteratively refines poses using epipolar geometry constraints—without requiring network adaptation or scene re-reconstruction. Its modular architecture enables flexible substitution of feature extraction and matching components, eliminating dependence on specialized descriptors or networks. Evaluated on the 7-Scenes and Cambridge Landmarks benchmarks, GS-SMC reduces translational error by 53.3% and 40.7%, and rotational error by 56.9% and 53.2%, respectively—outperforming state-of-the-art approaches significantly.
📝 Abstract
Camera pose refinement aims at improving the accuracy of initial pose estimation for applications in 3D computer vision. Most refinement approaches rely on 2D-3D correspondences with specific descriptors or dedicated networks, requiring reconstructing the scene again for a different descriptor or fully retraining the network for each scene. Some recent methods instead infer pose from feature similarity, but their lack of geometry constraints results in less accuracy. To overcome these limitations, we propose a novel camera pose refinement framework leveraging 3D Gaussian Splatting (3DGS), referred to as GS-SMC. Given the widespread usage of 3DGS, our method can employ an existing 3DGS model to render novel views, providing a lightweight solution that can be directly applied to diverse scenes without additional training or fine-tuning. Specifically, we introduce an iterative optimization approach, which refines the camera pose using epipolar geometric constraints among the query and multiple rendered images. Our method allows flexibly choosing feature extractors and matchers to establish these constraints. Extensive empirical evaluations on the 7-Scenes and the Cambridge Landmarks datasets demonstrate that our method outperforms state-of-the-art camera pose refinement approaches, achieving 53.3% and 56.9% reductions in median translation and rotation errors on 7-Scenes, and 40.7% and 53.2% on Cambridge.