CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds

2026-08-01

AI Summary

This work addresses the challenge of preserving fine-grained subject-specific details in subject-driven image generation, which is hindered by the scarcity of high-quality paired supervision data. To this end, we propose CopyCat, a lightweight fine-tuning framework that requires only a single proxy image and optimizes a pre-trained diffusion model in seconds through a one-shot adaptation, significantly enhancing fine-grained subject consistency. At its core is the Fine-grained Consistency LoRA (FCLoRA), which integrates a self-reconstruction objective and adapts solely the visual stream within a dual-stream diffusion Transformer. Notably, our method operates without additional training data and generalizes effectively to unseen subjects and prompts. Extensive experiments on DreamBench and XVerseBench demonstrate that CopyCat substantially outperforms existing approaches in both single- and multi-subject scenarios.

Abstract

Recent subject-to-image models have achieved impressive progress in personalized image generation, yet they still struggle to preserve fine-grained subject-specific details. A major reason is the lack of high-quality fine-grained identity supervision: real paired data are expensive to collect, while synthesized training pairs often preserve only coarse subject appearance and fail to capture subtle subject-specific details. In this work, we propose CopyCat, a lightweight model-refinement framework that improves fine-grained subject consistency within only a few seconds. CopyCat performs a one-time refinement of a pretrained subject-to-image model by attaching a lightweight Fine-grained Consistency LoRA (FCLoRA) and optimizing it using a single proxy image, which is used as both the conditioning image and the reconstruction target. This exact self-reconstruction objective substantially simplifies the optimization task, enabling effective fine-grained refinement within only a few seconds. The refinement is performed only once; the resulting model can be directly applied to diverse unseen reference subjects and prompts without further subject-specific optimization. We further revisit subject-to-image LoRA training in double-stream diffusion transformers and find that adapting only the visual stream consistently improves subject consistency. Extensive experiments on DreamBench and XVerseBench demonstrate consistent improvements in fine-grained subject consistency across representative subject-to-image models under both single- and multi-subject settings.

Problem

• fine-grained subject consistency
• subject-to-image models
• personalized image generation
• identity supervision
• subject-specific details

Innovation

• CopyCat
• Fine-grained Consistency
• LoRA
• Subject-to-Image Generation
• Self-Reconstruction