transfer learning

This study investigates whether explicit domain adaptation consistently improves cross-domain sentiment analysis performance when the backbone of a pretrained language model is frozen. We train lightweight MLP adapters on frozen backbones of varying scales and domain specialization—including Qwen3-Embedding (0.6B/4B/8B), RoBERTa-base, and FinBERT—and evaluate them with alignment strategies such as DANN, MMD, and supervised contrastive learning (SCL) on tasks transferring from consumer reviews to movie reviews and financial news. Our findings reveal that the efficacy of domain adaptation heavily depends on whether the backbone already encodes target-domain knowledge. Adversarial alignment (e.g., DANN) can degrade performance on domain-specialized backbones like FinBERT, whereas SCL proves more robust. While adaptation yields negligible gains on SST-2, it substantially recovers performance for smaller general-purpose backbones on financial subsets.

Domain adaptation (DA) faces practical challenges including difficulty in problem identification and lack of principled guidance for method selection. Method: This paper proposes the first problem-oriented DA framework, introducing a novel five-dimensional scenario taxonomy that systematically characterizes the causes and patterns of data distribution shift, complemented by a scenario identification guide and a method recommendation mechanism. Grounded in design science research, the framework undergoes iterative empirical evaluation across synthetic and real-world datasets, as well as a 100-participant user study. Contribution/Results: Results demonstrate significant improvements in interpretability, generality, and usability. The framework substantially enhances non-expert users’ accuracy in understanding DA tasks and their rationality in method selection, thereby addressing a critical gap in problem-driven adaptive decision-making research.

Conventional transfer learning assumes structural homogeneity between source and target models (e.g., regression → regression), limiting applicability to cross-task settings (e.g., regression → classification) and small-sample regimes where both source and target data are scarce. Method: We propose a decoupled transfer learning framework that explicitly separates representation sufficiency from task-structure dependency. It jointly estimates sufficient statistics and learns invariant representations from the source domain, then augments them with target-domain independent components—enabling theoretically grounded representation enhancement without requiring model homomorphism. Contribution/Results: The framework guarantees theoretical sufficiency and consistency of learned representations. It supports heterogeneous task transfer and few-shot knowledge adaptation. Empirical evaluation on synthetic and real-world benchmarks demonstrates significant improvements in generalization performance, particularly under extreme target-data scarcity—outperforming state-of-the-art homomorphic and adversarial transfer methods.

This work addresses the challenge of selecting appropriate source domains and pre-trained models for unsupervised domain adaptation when target-domain labels are unavailable—a critical yet underexplored problem that often limits adaptation performance. To tackle this, the authors propose the PAS (Pre-trained Adaptation Suitability) scoring mechanism, which evaluates source–target domain compatibility and model transferability by analyzing the geometry of pre-trained feature embedding spaces. Remarkably, PAS accurately predicts post-adaptation target accuracy without requiring any target labels. This approach enables, for the first time, joint unsupervised selection of both source domains and pre-trained models. Extensive experiments on multiple image classification benchmarks demonstrate a strong correlation between PAS scores and actual adaptation accuracy, leading to significantly improved performance while substantially reducing computational overhead.

Existing domain adaptation methods (e.g., DSN) separate domain-invariant representations (DIReps) from domain-specific representations (DDReps) via orthogonality constraints. However, weak orthogonality often leaves discriminative information in DDReps—causing “information leakage” that degrades target-domain generalization. This paper proposes a KL-divergence-driven DDRep minimization mechanism that explicitly compresses the DDRep distribution within deep networks, thereby enhancing the purity and transferability of DIReps. The constraint is compatible with pretrained models and enables end-to-end domain-separation learning. Evaluated on standard image benchmarks (e.g., Office), the method achieves or surpasses state-of-the-art performance. Synthetic-data experiments demonstrate robustness to initialization and substantial improvements in cross-domain classification accuracy. The core innovation lies in replacing conventional orthogonality constraints with a distribution-level KL-divergence penalty, fundamentally mitigating representation leakage.

This work addresses the lack of rigorous theoretical foundations for domain alignment mechanisms in semi-supervised domain adaptation. We propose the first unified generalization error analysis framework, characterizing the sample complexity of deep domain adaptation networks based on both Maximum Mean Discrepancy (MMD) and adversarial objectives. Methodologically, we jointly model feature transformation and classifier learning, quantify function class complexity via covering number theory, and integrate MMD and adversarial losses to derive a data-dependent generalization upper bound. Theoretically, we establish that sample complexity grows quadratically with network depth and width, and that target-domain risk scales with the inverse square root of labeled target samples—thereby enhancing robustness under limited labeling. Empirical evaluations validate these theoretical predictions, providing an interpretable, principled foundation for modern deep domain adaptation methods.

This work addresses the challenge of lacking identifiability guarantees in unsupervised multi-source domain adaptation with high-dimensional data by proposing a general domain adaptation framework grounded in Markov blanket structure. The method learns compact latent representations that capture task-relevant distribution shifts and leverages the Markov blanket of the label—comprising its parents, children, and spouses—to guide identifiable representation learning. It reveals for the first time that representations relying solely on complete predictive information are underdetermined under general settings, thereby circumventing strong assumptions commonly required by existing approaches, such as independent latent variables or invariant label distributions. By integrating causal representation learning with nonparametric deep models, the framework achieves theoretically guaranteed identifiability and significantly improves target-domain generalization across diverse distribution shift scenarios.

This paper addresses cross-domain few-shot learning under frozen backbone networks (CNNs or Vision Transformers), proposing a general, fine-tuning-free solution. Methodologically, it reformulates few-shot classification as a multi-instance verification (MIV) task—the first such formulation—and introduces a lightweight, backbone-agnostic MIV-head module. This module operates solely during meta-testing, performing pairwise similarity verification between target-domain support and query instances without updating any backbone parameters. Evaluated on the extended Meta-dataset benchmark, our approach matches the performance of leading adaptation methods while substantially outperforming conventional linear classification heads; notably, its adaptation overhead is reduced by an order of magnitude. The work establishes an efficient, plug-and-play paradigm for few-shot transfer learning with black-box feature extractors, enabling rapid domain adaptation without backbone modification.

This work addresses the challenge of limited training data in image reconstruction tasks, particularly in cross-domain scenarios such as undersampled MRI, where performance is often constrained. The authors propose a two-stage transfer learning framework: first, a universal feature extractor is learned via bilevel optimization from heterogeneous multi-source data—including diverse anatomical structures, sampling rates, and natural images—and subsequently, a lightweight adapter is trained for the target domain. By uniquely integrating bilevel optimization with cross-domain transfer, this approach establishes a transferable regularization mechanism for reconstruction. It achieves significantly improved image quality under few-shot conditions and demonstrates strong generalization across varying anatomical structures, imaging modalities, and sampling rates.

In few-shot domain adaptation, two key challenges hinder performance: (1) hyperparameter tuning is impractical due to scarce validation data, and (2) models lack robustness under distribution shift. To address these, we propose Soup-Adapter—a hyperparameter-agnostic, distributionally robust adapter ensemble method. Our approach extends the CLIP adapter paradigm to DINOv2 for the first time and introduces a reparameterizable adapter soup: multiple adapters are trained independently via distinct paths, their outputs are averaged at inference, and their parameters are concatenated and reparameterized to stabilize optimization. Experiments demonstrate that Soup-Adapter consistently outperforms single-adapter baselines across a wide range of hyperparameters and significantly improves both accuracy and robustness on cross-domain few-shot tasks. This work establishes a new, efficient, practical, and generalizable paradigm for few-shot domain adaptation.