Global Variational Inference Enhanced Robust Domain Adaptation

2025-07-04

AI Summary

Existing deep domain adaptation methods rely on mini-batch training, hindering global distribution modeling and resulting in unstable cross-domain alignment and poor generalization. To address this, we propose a continuous class-conditional global prior modeling framework: (i) structural-aware class-conditional priors are learned via variational inference; (ii) a stochastic sampling codebook is introduced to prevent posterior collapse; (iii) high-confidence pseudo-label filtering and target-domain sample generation jointly enhance alignment robustness; and (iv) latent reconstruction combined with diffusion-model-based contrastive optimization improves noise tolerance. The method achieves state-of-the-art performance across 38 transfer tasks on four benchmark datasets. Theoretical analysis demonstrates significant improvements in the tightness of the ELBO lower bound, continuity of the learned priors, and distributional robustness.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising

Abstract

Deep learning-based domain adaptation (DA) methods have shown strong performance by learning transferable representations. However, their reliance on mini-batch training limits global distribution modeling, leading to unstable alignment and suboptimal generalization. We propose Global Variational Inference Enhanced Domain Adaptation (GVI-DA), a framework that learns continuous, class-conditional global priors via variational inference to enable structure-aware cross-domain alignment. GVI-DA minimizes domain gaps through latent feature reconstruction, and mitigates posterior collapse using global codebook learning with randomized sampling. It further improves robustness by discarding low-confidence pseudo-labels and generating reliable target-domain samples. Extensive experiments on four benchmarks and thirty-eight DA tasks demonstrate consistent state-of-the-art performance. We also derive the model's evidence lower bound (ELBO) and analyze the effects of prior continuity, codebook size, and pseudo-label noise tolerance. In addition, we compare GVI-DA with diffusion-based generative frameworks in terms of optimization principles and efficiency, highlighting both its theoretical soundness and practical advantages.

Problem

• Enhances domain adaptation with global variational inference
• Mitigates domain gaps via latent feature reconstruction
• Improves robustness by filtering unreliable pseudo-labels

Innovation

• Global variational inference for domain adaptation
• Latent feature reconstruction minimizes domain gaps
• Global codebook learning mitigates posterior collapse