XCalib Depth-Guided Geometric Optimization for Dense Thermal-Visible Video Registration

2026-09-18

AI Summary

本文提出XCalib框架，通过几何优化和深度预测解决热可见视频配准中的空间错位问题，提高对准精度和时间稳定性。

Abstract

Image registration is a vital preprocessing step in multimodal perception tasks, including image fusion, object detection, and semantic segmentation. In Advanced Driver- Assistance Systems (ADAS), spatial misalignment between visible (RGB) and infrared (IR) cameras -caused by non-coincident optical axes and field-of-view differences- introduces non-uniform parallax and visual ghosting. Classical keypoint-based methods are restricted to global homographies that fail under dynamic depth, while unconstrained dense flow algorithms lack structural regularization and suffer from temporal instability. In this paper, we propose XCalib, an unsupervised dense thermal-visible registration framework that bridges this gap. Rather than serving as an absolute metric calibration tool, XCalib leverages virtual pinhole camera parameterization strictly as a geometric constraint space. By optimizing effective relative pose and intrinsics alongside predicted monocular metric depth, XCalib restricts the search space of spatial displacements to physically valid projection geometries. Our key contributions are: (1) a novel registration paradigm that uses camera parameterization as an implicit regularizer for dense cross-modal warping; (2) Normalized Edges Correlation (NEC), a robust structural similarity metric tailored to cross- spectral alignment; and (3) extensive quantitative and qualitative evaluations across public ADAS datasets, demonstrating superior temporal stability and alignment accuracy over unconstrained dense flow baselines.

Problem

• Image Registration
• Multimodal Perception
• Advanced Driver-Assistance Systems
• Spatial Misalignment
• Temporal Instability

Innovation

• XCalib
• camera parameterization
• Normalized Edges Correlation (NEC)
• unsupervised dense registration