Multimodal domain adaptation under label shift and blockwise missing modalities

2026-08-02

AI Summary

This work addresses the challenge in multimodal domain adaptation where label distribution shift and block-wise modality missing coexist, rendering conventional alignment strategies ineffective. To overcome this, the authors propose a reference-anchored domain adaptation framework that leverages a reference modality common to all domains to estimate the target-domain label distribution, reweights source-domain samples accordingly, and aligns auxiliary modalities to the target-domain representation via canonical correlation analysis (CCA) and ridge regression mapping. Prediction information is transferred through a density ratio model, and a proxy-label assistance mechanism is introduced to handle scenarios with scarce true labels. This study establishes the first effective alignment approach under such complex conditions and develops a unified rotation-matching theory linking CCA and ridge regression, significantly improving prediction stability and calibration on both synthetic and renal cell carcinoma datasets.

Abstract

Multimodal domain adaptation uses labeled source datasets to predict outcomes in an unlabeled target population. Here, different sources may observe different subsets of modalities and have distribution shifts from the target. In this scenario, the cross-modal patterns used for alignment depend on the outcome distribution, so aligning unadjusted source data can misrepresent the target. Nevertheless, the distribution shift of the outcome cannot be measured directly when labels are unavailable in the target sample and the missing modality blocks prevent simple pooling. We propose a reference-anchored domain adaptation method. A reference modality observed in every source and the target is used to estimate the target outcome distribution and reweight source observations before alignment. The auxiliary modalities are then mapped to a common, target-defined representation obtained by canonical correlation analysis (CCA) in the target and reproduced in each source by ridge-regression maps. Outcome information is transferred through density-ratio models on the aligned representation. When gold-standard source labels are sparse, a surrogate-label-assisted approach is developed to enable robust domain adaptation. We establish a unified-rotation match-up result for the target CCA and source ridge maps and consistency of the target conditional outcome distribution. Simulations and a renal cell carcinoma (RCC) application show improved calibration and stable prediction under distributional shift and blockwise missing modalities.

Problem

• multimodal domain adaptation
• label shift
• blockwise missing modalities
• distribution shift
• unlabeled target

Innovation

• multimodal domain adaptation
• label shift
• blockwise missing modalities
• canonical correlation analysis
• reference-anchored alignment