Incentive Noise and Structural Prior Infusion for Multi-modal Object Re-Identification

2026-09-21

AI Summary

该研究针对多模态物体重识别中的噪声和结构差异问题，提出了一种结合正向激励噪声和结构先验注入的新框架，以增强跨模态语义表示的一致性和鲁棒性。

Abstract

Multi-modal object Re-Identification (ReID) benefits from complementary information across heterogeneous imaging modalities. To further enrich semantic representation, text descriptions have recently been incorporated as an additional modality. However, recent vision-language approaches often treat text descriptions as clean, deterministic signals and overlook their inherent noise, including modality-mismatched phrases and semantically ambiguous expressions. Moreover, prevailing methods lack explicit mechanisms to reconcile fine-grained structural discrepancies between modalities, even after high-level semantic alignment. To address these challenges, we propose a novel framework centered on Positive-Incentive Noise (π-noise) and structured prompt modulation. First, the Semantic Cross-Modal Modulator harnesses task-aware π-noise, sampled from a distribution conditioned on both visual and text inputs, to perturb global tokens and enable semantics-guided cross-modal compensation. Second, the Structure-Aware Prompt Adapter injects learnable geometric priors via prompts to enhance spatial consistency. Third, the Context-Aware Sparse Fusion module distills structural context to guide adaptive fusion while shielding identity features from noisy local details. Experiments on three multi-modal ReID benchmarks demonstrate the effectiveness and robustness of our approach. The code is available at https://github.com/zw-absin/INSPI.

Problem

• multi-modal object Re-Identification
• text descriptions
• inherent noise
• structural discrepancies
• semantic alignment

Innovation

• Positive-Incentive Noise
• Structured Prompt Modulation
• Semantic Cross-Modal Modulator
• Structure-Aware Prompt Adapter
• Context-Aware Sparse Fusion