FMRFusion: Frequency-Aware Multi-View Representation Learning for Heterogeneous Image Fusion

2026-06-06

AI Summary

Existing infrared and visible image fusion methods suffer from limited performance and robustness due to their reliance on single-module feature extraction, which fails to adequately model the distinct characteristics of heterogeneous modalities. To address this limitation, this work proposes a frequency-aware multi-view representation learning network that jointly optimizes local detail and global semantic representations through multi-scale structural awareness, bilinear frequency-domain decomposition, cross-view complementary interaction, and a flow-matching mechanism. The proposed approach achieves state-of-the-art and consistently stable fusion performance across multiple benchmark datasets, demonstrating significant superiority over existing methods—particularly in challenging scenarios such as nighttime conditions—thereby substantially enhancing the quality and robustness of heterogeneous image fusion.

Abstract

Infrared and visible image fusion aims to generate a composite image that retains significant target information and preserves detailed textures, integrating two heterogeneous modalities. Previous image fusion methods typically adopt a single-module stacking approach to extract features from the two modalities. However, these approaches may result in incomplete learning of their distinct characteristics, thereby limiting the fusion effectiveness and constrain ing robustness in real-world heterogeneous data scenarios. To address these challenges, we propose FMRFusion, a frequency-aware multi-view representation learning network for Heterogeneous Image Fusion. A Multi-Scale Struc tural Perception Module is introduced to effectively capture discriminative structures, extracting fine-grained local structures and essential contextual information. A bilinear frequency decomposition mechanism is employed to sepa rate features into high-frequency and low-frequency components, enabling joint modeling of local details and global representations across different frequency domains. Moreover, a Cross-View Complementary Interaction is incorpo rated to explicitly model and fuse the complementary characteristics between reflected light information and radiative intensity responses, facilitating effective cross-view interaction. We further improve the Performance of the fused results by flow matching, which progressively refines the fused features by learning the transformation from coarse data to high-quality representations. Extensive experiments conducted on multiple benchmark datasets demonstrate that FMRFusion achieves superior and consistent performance across a range of fusion tasks, especially in nighttime scenarios

Problem

• heterogeneous image fusion
• infrared and visible image fusion
• multi-view representation
• frequency-aware learning
• cross-modal fusion

Innovation

• Frequency-aware
• Multi-view representation learning
• Bilinear frequency decomposition
• Cross-view complementary interaction
• Flow matching