Step-wise Distribution Alignment Guided Style Prompt Tuning for Source-free Cross-domain Few-shot Learning

2024-11-15

arXiv.org

AI Summary

This paper addresses source-free cross-domain few-shot learning (SF-CDFSL)—a challenging setting where only a pre-trained language model and a few labeled samples from the target domain are available, with no access to source-domain data or training strategies. We propose a stepwise distribution alignment-guided style prompt tuning method. Our approach implicitly reduces inter-domain distribution discrepancy through outer-stage multi-step predictive distribution alignment and inner-stage joint classifier optimization. Coupled with a lightweight style prompt design, it significantly lowers computational overhead. Extensive experiments on five benchmark datasets demonstrate consistent superiority over existing prompt-tuning methods and state-of-the-art (SOTA) approaches. Ablation studies validate the effectiveness of each component. The code is publicly available.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Prompt Engineering / Prompting
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Existing cross-domain few-shot learning (CDFSL) methods, which develop source-domain training strategies to enhance model transferability, face challenges with large-scale pre-trained models (LMs) due to inaccessible source data and training strategies. Moreover, fine-tuning LMs for CDFSL demands substantial computational resources, limiting practicality. This paper addresses the source-free CDFSL (SF-CDFSL) problem, tackling few-shot learning (FSL) in the target domain using only pre-trained models and a few target samples without source data or strategies. To overcome the challenge of inaccessible source data, this paper introduces Step-wise Distribution Alignment Guided Style Prompt Tuning (StepSPT), which implicitly narrows domain gaps through prediction distribution optimization. StepSPT proposes a style prompt to align target samples with the desired distribution and adopts a dual-phase optimization process. In the external process, a step-wise distribution alignment strategy factorizes prediction distribution optimization into a multi-step alignment problem to tune the style prompt. In the internal process, the classifier is updated using standard cross-entropy loss. Evaluations on five datasets demonstrate that StepSPT outperforms existing prompt tuning-based methods and SOTAs. Ablation studies further verify its effectiveness. Code will be made publicly available at url{https://github.com/xuhuali-mxj/StepSPT}.

Problem

• Addresses source-free cross-domain few-shot learning without source data
• Proposes style prompt tuning to align target domain distributions
• Reduces computational demands of fine-tuning large pre-trained models

Innovation

• Style prompt tuning for domain alignment
• Step-wise distribution alignment strategy
• Dual-phase optimization without source data