Parameter-Dynamic Adaptive Fusion and Calibration Network for RGBT Tracking

2026-08-03

AI Summary

This work addresses the limitations of existing RGBT tracking methods, which rely on fixed-parameter fusion strategies and struggle to handle target appearance variations and modality quality fluctuations. To overcome these challenges, the paper proposes PAFCNet, the first approach to introduce a target-adaptive hypernetwork mechanism that dynamically generates parameters for both modality fusion and spatiotemporal calibration, enabling fine-grained control over multimodal information and temporal-spatial features. By leveraging template representations to drive parameter generation, PAFCNet flexibly adjusts the fusion process in response to changing conditions, significantly enhancing tracking robustness in complex scenarios. Extensive experiments demonstrate that the method achieves state-of-the-art performance across multiple RGBT benchmarks, effectively mitigating the adverse effects of modality interference and appearance changes.

Abstract

Existing RGBT trackers typically employ fusion functions with fixed parameters across different targets and scenarios. Although dynamic-architecture methods improve fusion flexibility by selecting among predefined operations, they still cannot adapt the fusion parameters to the evolving target state. To address these issues, we propose a Parameter-Dynamic Adaptive Fusion and Calibration Network (PAFCNet) for RGBT tracking. PAFCNet dynamically generates target-conditioned parameters for multimodal fusion and temporal calibration, enabling the tracking process to adapt to target appearance variations and modality quality fluctuations. Specifically, we introduce a Target-Adaptive Hypernetwork (TA-HyperNet) that leverages template representations, which preserve stable target identity and recent appearance changes with less background interference, to generate target-conditioned parameters for subsequent fusion and calibration. Based on TA-HyperNet, we design a target-aware parameter-dynamic fusion module that uses the generated parameters to modulate the fusion process. This enables the fusion module to adapt to changes in target appearance and complex scene conditions. Furthermore, since spatio-temporal information propagation may accumulate tracking noise, we propose a dynamic spatio-temporal calibration module that employs TA-HyperNet to generate calibration parameters for spatio-temporal tokens. By dynamically calibrating historical information before propagation, the module improves the reliability of temporal representations. Experimental results demonstrate that PAFCNet achieves competitive performance on multiple RGBT tracking benchmarks.

Problem

• RGBT tracking
• adaptive fusion
• parameter dynamics
• target appearance variation
• modality quality fluctuation

Innovation

• parameter-dynamic fusion
• target-adaptive hypernetwork
• RGBT tracking
• temporal calibration
• multimodal fusion