few-shot adaptation

This paper addresses cross-domain few-shot learning under frozen backbone networks (CNNs or Vision Transformers), proposing a general, fine-tuning-free solution. Methodologically, it reformulates few-shot classification as a multi-instance verification (MIV) task—the first such formulation—and introduces a lightweight, backbone-agnostic MIV-head module. This module operates solely during meta-testing, performing pairwise similarity verification between target-domain support and query instances without updating any backbone parameters. Evaluated on the extended Meta-dataset benchmark, our approach matches the performance of leading adaptation methods while substantially outperforming conventional linear classification heads; notably, its adaptation overhead is reduced by an order of magnitude. The work establishes an efficient, plug-and-play paradigm for few-shot transfer learning with black-box feature extractors, enabling rapid domain adaptation without backbone modification.

This work addresses the lack of a unified, rigorous, and realistic evaluation protocol in few-shot transfer learning, which has led to unreliable method comparisons. To this end, we introduce the FEWTRANS benchmark—comprising ten diverse datasets—and the Hyperparameter Ensemble (HPE) evaluation protocol, which effectively mitigates validation set hallucination under data scarcity. Using this framework, we systematically demonstrate for the first time that the choice of pretrained model is more critical than the complexity of the transfer algorithm. We also quantify the performance collapse of multimodal models in specialized domains due to linguistic rarity. Our analysis reveals that simple full-parameter fine-tuning consistently outperforms most sophisticated methods, owing to its ability to flexibly reshape distributed representations and high-level semantic features. The FEWTRANS benchmark is publicly released to provide the community with a reproducible evaluation standard.

Fine-tuning large-scale vision-language models (VLMs) for few-shot adaptation (FSA) is computationally prohibitive due to their scale and data scarcity. Method: We propose a two-stage parameter-efficient fine-tuning (PEFT) framework: Stage 1 learns a task-specific feature extractor via PEFT, decoupling generic representations from task-specific knowledge; Stage 2 freezes this extractor and trains only a lightweight linear classifier. We further uncover a novel biphasic dynamic of PEFT in FSA and introduce a selective inference mechanism—during testing, only the text encoder adapter for novel classes is activated, while base-class embeddings are reused from the pretrained model. Results: Evaluated across 11 datasets, 3 backbone architectures, and 2 FSA settings with fixed hyperparameters, our method matches or surpasses state-of-the-art performance and demonstrates significantly improved cross-scenario robustness.

To address the adaptation challenge of contrastive pre-trained vision-language models (e.g., CLIP) for few-shot classification, this paper proposes a lightweight and efficient fine-tuning method that updates only the final projection matrix of the visual encoder. Our key contributions are: (i) the first demonstration that optimizing solely this low-dimensional projection layer—without modifying the text encoder or introducing auxiliary modules—outperforms mainstream adaptation strategies; and (ii) the introduction of an L2-distance regularization between the pre-trained and fine-tuned projection matrices, which significantly enhances generalization and robustness. The method drastically reduces trainable parameters and computational overhead. It achieves state-of-the-art performance across 11 standard few-shot benchmarks and demonstrates superior results on challenging tasks including cross-domain transfer, base-to-novel class generalization, and test-time adaptation.

This work addresses the lack of rigorous theoretical foundations for domain alignment mechanisms in semi-supervised domain adaptation. We propose the first unified generalization error analysis framework, characterizing the sample complexity of deep domain adaptation networks based on both Maximum Mean Discrepancy (MMD) and adversarial objectives. Methodologically, we jointly model feature transformation and classifier learning, quantify function class complexity via covering number theory, and integrate MMD and adversarial losses to derive a data-dependent generalization upper bound. Theoretically, we establish that sample complexity grows quadratically with network depth and width, and that target-domain risk scales with the inverse square root of labeled target samples—thereby enhancing robustness under limited labeling. Empirical evaluations validate these theoretical predictions, providing an interpretable, principled foundation for modern deep domain adaptation methods.

Few-shot image generation suffers from alignment distortion due to structural distribution mismatch between source and target domains: conventional latent-space consistency constraints—either instance-level or distribution-level—are prone to either content distortion (if overly strict) or weakened transfer efficacy (if too loose), further exacerbated by distribution estimation bias under extreme target-sample scarcity. This paper proposes a two-level domain alignment framework that achieves structure-preserving cross-domain alignment within a self-rotating proxy feature space. Its core innovation is the Equivariant Feature Rotation (EFR) mechanism: learning differentiable rotation matrices within a parameterized Lie group to ensure equivariance and lossless feature transformation. Integrated with proxy-space construction, few-shot fine-tuning, and joint feature alignment optimization, the method reduces FID by over 25% across multiple benchmarks, significantly improving generation quality and diversity while mitigating content distortion and distributional bias.

This work addresses the challenge of selecting appropriate source domains and pre-trained models for unsupervised domain adaptation when target-domain labels are unavailable—a critical yet underexplored problem that often limits adaptation performance. To tackle this, the authors propose the PAS (Pre-trained Adaptation Suitability) scoring mechanism, which evaluates source–target domain compatibility and model transferability by analyzing the geometry of pre-trained feature embedding spaces. Remarkably, PAS accurately predicts post-adaptation target accuracy without requiring any target labels. This approach enables, for the first time, joint unsupervised selection of both source domains and pre-trained models. Extensive experiments on multiple image classification benchmarks demonstrate a strong correlation between PAS scores and actual adaptation accuracy, leading to significantly improved performance while substantially reducing computational overhead.

Fine-tuning pretrained models often degrades their domain generalization capability. To address this, we propose a joint parameter selection mechanism that updates only a sparse, cross-domain gradient-consistent subset of parameters during fine-tuning. Grounded in theoretical analysis linking parameter sparsity to generalization error, our method introduces a dual-operator gradient consistency filter to dynamically identify parameters with high generalization potential for update. This strategy integrates parameter-efficient fine-tuning with gradient-driven structured sparsity, preserving the model’s original domain generalization while enhancing task-specific adaptation. Extensive experiments on multiple standard domain generalization benchmarks demonstrate significant improvements over existing state-of-the-art methods, validating the effectiveness of our approach in balancing generalization and adaptability.

Existing theoretical frameworks struggle to explain why larger-scale pre-trained models substantially reduce sample complexity on downstream tasks. This work proposes a novel theoretical framework—termed “caulking”—inspired by parameter-efficient fine-tuning methods such as adapters, low-rank adaptation, and partial fine-tuning. It establishes, for the first time, a provable relationship between the scale of pre-trained models and the sample complexity of downstream tasks. By rigorously linking stronger pre-training capabilities to reduced data requirements in transfer learning, this study not only addresses a critical gap in current theoretical understanding but also provides a solid foundation for empirically observed scaling laws, demonstrating that enhanced pre-training capacity can significantly decrease the number of samples needed for effective downstream adaptation.