Look Up and Look Back: Hidden Attention and Latent Orientation in a Frozen Foundation Model for Panoramic SLAM

2026-08-01

AI Summary

This work addresses persistent challenges in monocular panoramic SLAM—such as camera tilt, scale drift, and erroneous loop closures under large rotational motions—by introducing HALO-SLAM. The method is the first to uncover two key cues latent in a frozen panoramic geometric foundation model: gravity direction implicitly encoded in intermediate tokens and revisit compatibility signals derived from cross-view attention. Leveraging these insights, HALO-SLAM achieves IMU-free spherical front-end alignment and a novel three-stage loop closure pipeline that integrates event-based retrieval, attention-guided filtering, and geometric verification. Evaluated across 125 sequences from five real-world panoramic datasets, the approach attains a 100% success rate and consistently outperforms existing baselines in absolute trajectory error (ATE), reducing errors by 30–88%.

Abstract

Monocular panoramic SLAM benefits from substantial visual overlap under large camera rotations, yet remains prone to errors caused by camera tilt, scale drift, and false loop closures. We show that a frozen panoramic geometry foundation model provides useful internal cues beyond its explicit geometric outputs: intermediate tokens encode gravity in the camera frame, while cross-view attention provides a compatibility cue for potential revisits. Building on these cues, we present HALO-SLAM. A gravity readout enables IMU-free spherical upright canonicalization. For loop closure, we introduce a cost-aware three-stage cascade combining DBoW2 event-level retrieval, attention-based compatibility filtering, and dense geometric validation through symmetric submap augmentation. Accepted revisits yield pixel-aligned 3D--3D correspondences in both local gauges, from which robust $\mathrm{Sim}(3)$ constraints are estimated and jointly optimized with sequential constraints in a global pose graph. Across 125 sequences from five real-world panoramic benchmarks, our method achieves \textbf{100\%} sequence success (\textbf{125/125}) under the stated criterion and the lowest ATE among the evaluated methods on all five benchmarks, reducing ATE by \textbf{30--88\%} relative to the best ERP-native baseline on each benchmark.

Problem

• panoramic SLAM
• camera tilt
• scale drift
• false loop closures
• monocular

Innovation

• frozen foundation model
• hidden attention
• latent orientation
• panoramic SLAM
• gravity-aware canonicalization