Current Injection Spiking Neural Network for Infrared and Visible Image Fusion

2026-07-22

AI Summary

This work addresses the challenge of infrared and visible image fusion, where conventional artificial neural networks (ANNs) incur high computational costs, while existing spiking neural networks (SNNs) struggle to preserve sub-threshold complementary information due to binary spiking, leading to ineffective multimodal detail fusion. To overcome this, the authors propose CIS-Fuse, introducing a novel current-injected spiking (CIS) operator that enables adaptive cross-modal fusion at the membrane potential level. By injecting one modality as a gating auxiliary current into the driving neuron of the other modality, information integration is achieved prior to spike generation. The method employs an asymmetric dual-branch architecture, bidirectional cross-modal fusion modules, and a channel-wise learnable injection strength mechanism. Evaluated on four fusion benchmarks and downstream detection and segmentation tasks, CIS-Fuse achieves fusion quality comparable to state-of-the-art ANN methods while reducing inference energy consumption by nearly an order of magnitude compared to the ANN-based DCEvo model.

Abstract

Infrared and visible image fusion (IVIF) integrates the complementary information of two modalities into a single image with richer scene content. While existing methods are largely built on artificial neural networks (ANNs), which densely compute over all activations, spiking neural networks (SNNs) communicate through sparse binary spikes and compute only where and when a spike occurs, offering a route to more energy-efficient fusion. However, directly applying SNNs to IVIF creates a fundamental tension: cross-modal fusion relies on fine-grained responses from both modalities, whereas binary spikes can discard complementary cues that remain below the firing threshold. The membrane potential retains these subthreshold responses before firing, letting both modalities jointly shape the output when integrated at this stage. Building on this, we propose CIS-Fuse, a spiking network that performs cross-modal fusion directly at the membrane-potential level. At its core is the current injection spiking (CIS) operator, which injects one modality as a gated auxiliary current into the driving neuron of the other, so the two integrate before spike firing, with a per-channel learnable injection strength that adaptively regulates the modulation magnitude. Building on CIS, we construct a bidirectional cross-modal fusion (BCMF) module and deploy it on a dual-branch architecture with asymmetric stacking depths, where the two branches develop a clear functional specialization. Extensive experiments on four IVIF benchmarks and on downstream detection and segmentation show that CIS-Fuse achieves fusion quality on par with state-of-the-art ANN-based methods while inheriting the energy efficiency of spike-based computation, with roughly an order of magnitude lower inference energy than the similarly-sized ANN-based DCEvo. Code will be released upon publication.

Problem

• Infrared and Visible Image Fusion
• Spiking Neural Networks
• Cross-modal Fusion
• Membrane Potential
• Energy Efficiency

Innovation

• Spiking Neural Networks
• Membrane Potential Fusion
• Current Injection
• Cross-modal Integration
• Energy-efficient Computing