UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging

2026-07-23

AI Summary

This work addresses the challenge of cross-modal knowledge transfer in medical imaging under the practical constraint of unpaired multi-modal data. The authors propose UnDA, a novel framework that enables robust knowledge distillation and domain alignment without requiring paired samples. By leveraging anchor-guided extraction of semantic, structured class tokens, the method integrates attention pooling, an uncertainty-weighted optimal transport mechanism (UCT-OT) based on prediction confidence, and prototype-based contrastive learning (ProtoNCE) to dynamically suppress noisy predictions from the source domain and enhance inter-class consistency. Key innovations include a backbone-agnostic alignment module, the confidence-aware UCT-OT strategy, and a memory bank for maintaining class prototypes. Extensive experiments demonstrate that UnDA significantly improves segmentation accuracy and boundary precision on the target modality under strict unpaired settings, validating its effectiveness in transferring structured knowledge across heterogeneous imaging modalities.

Abstract

Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions. To overcome these challenges, we propose UnDA, an anchor-guided framework for unpaired cross-modal distillation. Our approach introduces a backbone-agnostic Alignment Module that extracts semantically structured class tokens via an attention based pooling mechanism. To ensure robust knowledge transfer, we propose Uncertainty-Weighted Optimal Transport (UCT-OT), which dynamically weights feature-level alignment based on prediction confidence, effectively suppressing noisy supervision. Furthermore, a per-class ProtoNCE objective maintains stable prototype memories to enforce global discriminability across unpaired batches. Evaluations on representative segmentation tasks under strictly unpaired settings show consistent improvements in accuracy and boundary precision in the target modality, demonstrating that meaningful structural knowledge can be transferred across heterogeneous data sources without paired datasets.

Problem

• unpaired domain alignment
• cross-modal knowledge transfer
• medical imaging
• modality gap
• noisy supervision

Innovation

• Unpaired Domain Alignment
• Cross-Modal Knowledge Distillation
• Uncertainty-Weighted Optimal Transport
• Attention-based Pooling
• ProtoNCE