CRT-HMAR: Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation for Open-Task-Aware Infrared-Visible Image Fusion

2026-10-06

AI Summary

This study addresses the limited generalizability of existing infrared-visible image fusion methods to unseen tasks, which constrains their deployment in open-world scenarios. To this end, this work proposes a causal demand tracing-guided hierarchical multi-agent regulation framework for open-task-aware fusion. The core innovation lies in shifting the modeling paradigm from task-oriented to demand-oriented by constructing an "explain-balance-mitigate" hierarchical regulation chain. This architecture integrates causal demand tracing, history-informed multi-objective balancing, and inter-level conflict mitigation techniques. Experimental results demonstrate that the proposed method significantly enhances generalization to unseen tasks across five downstream applications while effectively preserving performance and equilibrium on known tasks.

Abstract

Infrared and visible (IR-VIS) image fusion integrates complementary multimodal information into a single fused image to support downstream vision tasks. However, existing methods are typically tailored to seen tasks within a fixed task set and struggle to generalize to unseen tasks, which restricts their applicability in real-world open-task scenarios. To address this issue, this paper proposes CRT-HMAR, a Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation Framework for open-task-aware IR-VIS image fusion. CRT-HMAR introduces a Causal Requirement Tracing Task Localization mechanism, which actively intervenes in key image information and observes task-network response variations to map task-specific semantic preferences into image-level causal requirement maps. Based on these maps, a requirement analysis agent aggregates task-specific requirement knowledge to adaptively guide requirement-customized image fusion. Moreover, CRT-HMAR incorporates History-Analysis Multi-Objective Balancing and Task-Level-Correction Conflict Mitigation mechanisms, jointly constructing a hierarchical regulation chain of "requirement interpretation - task balancing - conflict mitigation". Through multiple collaborative agents, CRT-HMAR dynamically regulates key processes including open-task requirement modeling, multi-task balanced optimization, and gradient conflict mitigation. Extensive experiments on open-task scenarios involving five downstream tasks demonstrate that CRT-HMAR significantly improves generalization to unseen tasks while maintaining the performance and balance of seen tasks. Overall, CRT-HMAR shifts IR-VIS image fusion from task-oriented modeling toward requirement-oriented modeling, promoting its extension from closed-task settings to real-world open-task scenarios.

Problem

• Infrared-Visible Image Fusion
• Open-Task Generalization
• Unseen Tasks
• Downstream Vision Tasks

Innovation

• Causal Requirement Tracing
• Hierarchical Multi-Agent Regulation
• Open-Task-Aware Fusion
• Infrared-Visible Image Fusion
• Multi-Objective Balancing