🤖 AI Summary
To address pervasive edge blurring, semantic-structural distortion, and degraded downstream detection performance in infrared and visible-light image fusion, this paper identifies insufficient semantic-structural feature modeling in existing learning-based methods as the root cause of structural inconsistency in fused results. We propose a Semantic Structure-Preserving Fusion (SSPF) framework: (1) a novel Structure Feature Extractor (SFE) explicitly encodes structural semantics from source images; (2) a multi-scale Structure-Preserving Fusion (SPF) module jointly optimizes cross-modal structural consistency; and (3) a semantic consistency constraint enforces alignment between the fused image and source images in both high-level semantics and geometric structure. Extensive experiments on three benchmarks—TNO, INO, and RoadScene—demonstrate state-of-the-art performance, surpassing eight leading methods across quantitative metrics (e.g., EN, SD, SSIM) and downstream detection accuracy (e.g., mAP of YOLOv8).
📝 Abstract
Most existing learning-based infrared and visible image fusion (IVIF) methods exhibit massive redundant information in the fusion images, i.e., yielding edge-blurring effect or unrecognizable for object detectors. To alleviate these issues, we propose a semantic structure-preserving approach for IVIF, namely SSPFusion. At first, we design a Structural Feature Extractor (SFE) to extract the structural features of infrared and visible images. Then, we introduce a multi-scale Structure-Preserving Fusion (SPF) module to fuse the structural features of infrared and visible images, while maintaining the consistency of semantic structures between the fusion and source images. Owing to these two effective modules, our method is able to generate high-quality fusion images from pairs of infrared and visible images, which can boost the performance of downstream computer-vision tasks. Experimental results on three benchmarks demonstrate that our method outperforms eight state-of-the-art image fusion methods in terms of both qualitative and quantitative evaluations. The code for our method, along with additional comparison results, will be made available at: https://github.com/QiaoYang-CV/SSPFUSION.
