Multi-Modal Image Fusion via Intervention-Stable Feature Learning

2026-03-24

AI Summary

Existing multimodal image fusion methods are highly susceptible to spurious correlations present in training datasets, leading to a significant drop in generalization under distribution shifts. To address this limitation, this work introduces causal intervention into multimodal fusion for the first time, proposing an intervention framework grounded in Pearl’s causal hierarchy. The framework employs three strategies—complementary masking, random region masking, and modality dropout—to identify robust inter-modal dependencies. Furthermore, a Causal Feature Integrator (CFI) and an adaptive invariance gating mechanism are designed to prioritize the fusion of stable, causally invariant features. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance across multiple public benchmarks and downstream high-level vision tasks, substantially enhancing model robustness under distribution shifts.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-modal image fusion integrates complementary information from different modalities into a unified representation. Current methods predominantly optimize statistical correlations between modalities, often capturing dataset-induced spurious associations that degrade under distribution shifts. In this paper, we propose an intervention-based framework inspired by causal principles to identify robust cross-modal dependencies. Drawing insights from Pearl's causal hierarchy, we design three principled intervention strategies to probe different aspects of modal relationships: i) complementary masking with spatially disjoint perturbations tests whether modalities can genuinely compensate for each other's missing information, ii) random masking of identical regions identifies feature subsets that remain informative under partial observability, and iii) modality dropout evaluates the irreplaceable contribution of each modality. Based on these interventions, we introduce a Causal Feature Integrator (CFI) that learns to identify and prioritize intervention-stable features maintaining importance across different perturbation patterns through adaptive invariance gating, thereby capturing robust modal dependencies rather than spurious correlations. Extensive experiments demonstrate that our method achieves SOTA performance on both public benchmarks and downstream high-level vision tasks.

Problem

• multi-modal image fusion
• spurious correlations
• distribution shifts
• robustness
• causal dependencies

Innovation

• intervention-stable learning
• causal feature integration
• multi-modal image fusion
• distributional robustness
• invariance gating