BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion

2026-07-27

AI Summary

This work addresses the cross-sensor misalignment in mobile infrared–visible imaging caused by disparities in optics, viewpoint, field of view, and exposure timing. To overcome this challenge without requiring explicit calibration, the authors propose BeyondFusion, a unified latent diffusion framework that integrates a Cross-Modal Self-Alignment (CMSA) module within a denoising U-Net. This module dynamically learns adaptive correspondences between infrared and visible modalities in a shared attention space, while a misalignment augmentation strategy enables joint infrared super-resolution and multimodal fusion within a single generative process. By eliminating the need for explicit registration or geometric transformation, the method substantially transcends the limitations of traditional calibration-dependent approaches. Experiments demonstrate strong generalization on both public benchmarks and real-world mobile systems under unaligned, low-resolution, and asynchronously captured conditions, with downstream pedestrian detection tasks confirming its practical utility.

Abstract

Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cross-sensor misalignment caused by different optics, viewpoints, fields of view, and exposure timings hinders practical deployment. In this paper, we propose BeyondFusion, a unified latent diffusion framework for calibration-free visible-guided infrared super-resolution and infrared-visible fusion tasks. The proposed framework supports both task-specific training and joint training where two tasks are optimized and executed as two readouts of the same generative process. Instead of relying on explicit registration or geometric warping, BeyondFusion introduces a cross-modal self-aligning (CMSA) module into the denoising U-Net. CMSA reorganizes infrared and visible latent tokens into a shared attention space to learn content-adaptive cross-modal correspondence during the denoising process. Together with misalignment augmentation module, the model is facilitated to exploit visible structural and semantic cues while preserving thermal consistency, enabling high-frequency infrared reconstruction and informative fused-image generation under uncalibrated conditions. Extensive experiments on public benchmarks and a mobile infrared-visible imaging system show strong performance across aligned inputs, low-resolution infrared observations, synthetic misalignments, and real mobile captures with unsynchronized sensors. Ablation studies, unified training analysis, and downstream pedestrian detection further validate the effectiveness of BeyondFusion for calibration-free multimodal imaging.

Problem

• infrared-visible fusion
• super-resolution
• cross-sensor misalignment
• calibration-free imaging
• multimodal imaging

Innovation

• latent diffusion
• calibration-free
• cross-modal self-alignment
• infrared super-resolution
• infrared-visible fusion