One-shot Embroidery Customization via Contrastive LoRA Modulation

2025-09-23

AI Summary

Existing single-image style transfer methods struggle with fine-grained visual feature disentanglement and high-fidelity reproduction for embroidery—a textile art characterized by intricate stitch structures and material-specific properties. To address this, we propose a disentangled diffusion-based framework for embroidery style customization. First, we construct an image-analogy structure to explicitly separate content and style representations. Second, we design a two-stage contrastive LoRA modulation mechanism coupled with self-knowledge distillation, enabling precise style-content disentanglement from only one reference image. Built upon a pre-trained diffusion model, our method integrates LoRA-based low-rank adaptation, contrastive learning, and distillation into an end-to-end transfer pipeline. Evaluated on a newly curated embroidery benchmark, our approach significantly outperforms state-of-the-art methods. Moreover, it demonstrates strong generalization across diverse tasks—including artistic style transfer, line-art coloring, and appearance transfer—highlighting its robustness and versatility.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Diffusion models have significantly advanced image manipulation techniques, and their ability to generate photorealistic images is beginning to transform retail workflows, particularly in presale visualization. Beyond artistic style transfer, the capability to perform fine-grained visual feature transfer is becoming increasingly important. Embroidery is a textile art form characterized by intricate interplay of diverse stitch patterns and material properties, which poses unique challenges for existing style transfer methods. To explore the customization for such fine-grained features, we propose a novel contrastive learning framework that disentangles fine-grained style and content features with a single reference image, building on the classic concept of image analogy. We first construct an image pair to define the target style, and then adopt a similarity metric based on the decoupled representations of pretrained diffusion models for style-content separation. Subsequently, we propose a two-stage contrastive LoRA modulation technique to capture fine-grained style features. In the first stage, we iteratively update the whole LoRA and the selected style blocks to initially separate style from content. In the second stage, we design a contrastive learning strategy to further decouple style and content through self-knowledge distillation. Finally, we build an inference pipeline to handle image or text inputs with only the style blocks. To evaluate our method on fine-grained style transfer, we build a benchmark for embroidery customization. Our approach surpasses prior methods on this task and further demonstrates strong generalization to three additional domains: artistic style transfer, sketch colorization, and appearance transfer.

Problem

• Transferring fine-grained embroidery style features from single reference images
• Disentangling intricate stitch patterns and material properties for customization
• Overcoming limitations of existing style transfer methods on textile art

Innovation

• Contrastive learning framework disentangles fine-grained features
• Two-stage LoRA modulation captures style with single reference
• Self-knowledge distillation decouples style and content representations