A Scene is Worth a Thousand Features: Feed-Forward Camera Localization from a Collection of Image Features

2025-10-01

AI Summary

To address the high computational cost and trade-off between efficiency and accuracy in visual localization scene mapping, this paper proposes FastForward—the first end-to-end, single-pass feedforward framework for joint scene mapping and relocalization. Its core innovation lies in constructing a compact 3D anchor-based feature representation, enabling instantaneous pose estimation from a query image to the 3D scene via joint optimization of depth-aware feature extraction, image retrieval, and correspondence prediction networks. FastForward requires only one forward pass for localization, drastically reducing map construction time—by one to two orders of magnitude compared to state-of-the-art methods—while achieving leading accuracy on large-scale outdoor benchmarks. Moreover, it demonstrates strong generalization and robustness on unseen scenes, without requiring fine-tuning or additional adaptation.

Technology Category

• Computer Vision: Representation Learning for Vision
• Intelligent Robots: Localization, Mapping, and Navigation
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Visually localizing an image, i.e., estimating its camera pose, requires building a scene representation that serves as a visual map. The representation we choose has direct consequences towards the practicability of our system. Even when starting from mapping images with known camera poses, state-of-the-art approaches still require hours of mapping time in the worst case, and several minutes in the best. This work raises the question whether we can achieve competitive accuracy much faster. We introduce FastForward, a method that creates a map representation and relocalizes a query image on-the-fly in a single feed-forward pass. At the core, we represent multiple mapping images as a collection of features anchored in 3D space. FastForward utilizes these mapping features to predict image-to-scene correspondences for the query image, enabling the estimation of its camera pose. We couple FastForward with image retrieval and achieve state-of-the-art accuracy when compared to other approaches with minimal map preparation time. Furthermore, FastForward demonstrates robust generalization to unseen domains, including challenging large-scale outdoor environments.

Problem

• Achieving fast camera pose estimation with competitive accuracy
• Reducing lengthy mapping time through single feed-forward processing
• Enabling robust generalization to unseen large-scale environments

Innovation

• Single feed-forward pass for on-the-fly localization
• 3D-anchored feature collection representing mapping images
• Image-to-scene correspondence prediction for pose estimation