ConFusion: Continuous Fusion Space Learning for Fine-Grained Controllable Infrared and Visible Image Fusion

2026-07-26

AI Summary

Existing infrared and visible image fusion methods rely on discrete control conditions, limiting their ability to achieve fine-grained, region-aware adaptive fusion. To address this, this work proposes ConFusion, a novel framework that establishes a continuous fusion space for the first time, enabling user-intent-driven, instance-level controllable fusion. ConFusion employs a dual-branch architecture to disentangle modality-invariant and modality-specific representations, and integrates Gaussian conditional variables, Grounded SAM for instance segmentation, and a multimodal large language model to interpret textual instructions, thereby achieving semantically aligned mask-guided modulation. Experimental results demonstrate that ConFusion achieves state-of-the-art performance in both fusion quality and downstream task effectiveness, significantly enhancing the controllability and practical utility of fused outputs.

Abstract

Controllable infrared-visible image fusion aims to integrate complementary thermal and structural information with flexible region-aware modulation, producing fused images that adapt to diverse user requirements and downstream tasks. However, existing methods typically rely on predefined discrete control conditions, leading to a sparse space that fails to support fine-grained modulation demands. To address this, we propose ConFusion, a novel framework that learns the continuous fusion space via Gaussian-conditioned spatial-aware modulation, enabling instance-level fine-grained controllable infrared and visible image fusion. ConFusion employs a dual-branch architecture to disentangle modality-invariant and modality-specific representations under joint reconstruction and text-guided semantic alignment. Gaussian-conditioned instance modulation variables coupled with Grounded SAM-based instance masks guide instance-level fine-grained modulation through the Mask-Guided Specific Feature Modulator, while the Text-Driven Invariant Feature Enhancer improves semantic consistency and enhances fusion. During inference, the multimodal large language model parses user intents into instance-level modulation variables to guide image fusion. Extensive experiments show that ConFusion achieves state-of-the-art performance across multiple metrics in both fusion quality and downstream tasks, while supporting fine-grained controllable image fusion. Our code is available at https://github.com/HeyufeiAnto/Confusion

Problem

• infrared-visible image fusion
• fine-grained control
• continuous fusion space
• region-aware modulation
• controllable fusion

Innovation

• continuous fusion space
• fine-grained controllable fusion
• Gaussian-conditioned modulation
• instance-level modulation
• multimodal large language model