Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption

2025-01-18

AI Summary

Infrared–visible image fusion (IVIF) has long suffered from a lack of systematic surveys and faces persistent bottlenecks in data compatibility, perceptual fidelity, and downstream task adaptability. To address these challenges, we propose the first unified analytical framework that jointly emphasizes data alignment capability and task-driven generalization. Our framework establishes a multidimensional evaluation protocol encompassing registration robustness, fusion quality (measured by EN, SSIM, and VIF), and performance on high-level vision tasks. We further present the first comprehensive taxonomy of deep learning–based IVIF methods, accompanied by a comparative methodology table. Additionally, we open-source IVIF_ZOO—the most extensive, reproducible benchmark codebase to date. Extensive experiments demonstrate that our framework significantly improves algorithm selection efficiency and cross-task generalization. This work delivers an authoritative survey, a standardized, reproducible benchmark, and novel research directions for the IVIF community.

Technology Category

• Computer Vision: Image and Video Retrieval
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Infrared-visible image fusion (IVIF) is a critical task in computer vision, aimed at integrating the unique features of both infrared and visible spectra into a unified representation. Since 2018, the field has entered the deep learning era, with an increasing variety of approaches introducing a range of networks and loss functions to enhance visual performance. However, challenges such as data compatibility, perception accuracy, and efficiency remain. Unfortunately, there is a lack of recent comprehensive surveys that address this rapidly expanding domain. This paper fills that gap by providing a thorough survey covering a broad range of topics. We introduce a multi-dimensional framework to elucidate common learning-based IVIF methods, from visual enhancement strategies to data compatibility and task adaptability. We also present a detailed analysis of these approaches, accompanied by a lookup table clarifying their core ideas. Furthermore, we summarize performance comparisons, both quantitatively and qualitatively, focusing on registration, fusion, and subsequent high-level tasks. Beyond technical analysis, we discuss potential future directions and open issues in this area. For further details, visit our GitHub repository: https://github.com/RollingPlain/IVIF_ZOO.

Problem

• Infrared-Visible Image Fusion
• Image Quality
• Task Flexibility

Innovation

• Learning-based IVIF Methodology
• Image Quality Enhancement
• Task Adaptability and Fusion Strategies