🤖 AI Summary
To address the challenge of jointly modeling dynamic range and depth-of-field variations in multi-exposure and multi-focus image fusion, this paper proposes the first hierarchical text-guided fusion framework. Methodologically, we design a multi-granularity text encoder—capturing fine-grained details, mid-granularity structures, and coarse-granularity semantics—and build a hierarchical cross-modulation network. We further introduce a granularity-aware supervised loss and a saliency-driven semantic enhancement module to achieve precise cross-modal feature alignment and adaptive modulation. Our key innovation lies in explicitly embedding text granularity priors into the fusion process during training, eliminating the need for textual input at test time. Extensive experiments on mainstream multi-exposure and multi-focus benchmarks demonstrate consistent superiority over state-of-the-art methods, with significant improvements in PSNR and SSIM. The framework exhibits strong generalization across diverse fusion scenarios.
📝 Abstract
Image fusion aims to synthesize a single high-quality image from a pair of inputs captured under challenging conditions, such as differing exposure levels or focal depths. A core challenge lies in effectively handling disparities in dynamic range and focus depth between the inputs. With the advent of vision-language models, recent methods incorporate textual descriptions as auxiliary guidance to enhance fusion quality. However, simply incorporating coarse-grained descriptions hampers the understanding of fine-grained details and poses challenges for precise cross-modal alignment. To address these limitations, we propose Multi-grained Text-guided Image Fusion (MTIF), a novel fusion paradigm with three key designs. First, it introduces multi-grained textual descriptions that separately capture fine details, structural cues, and semantic content, guiding image fusion through a hierarchical cross-modal modulation module. Second, it involves supervision signals at each granularity to facilitate alignment between visual and textual features and enhance the utility of auxiliary text. Third, it adopts a saliency-driven enrichment module to augment training data with dense semantic content, further strengthening the cross-modal modulation and alignment. Extensive experiments show that MTIF consistently outperforms previous methods on both multi-exposure and multi-focus image fusion tasks.