Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization

2026-06-06

AI Summary

Current copyright protection mechanisms for diffusion models are vulnerable to adaptive attacks and struggle to prevent malicious forgery in personalized image generation. This work proposes a two-stage latent feature optimization method (TS-LFO), introducing for the first time a phased optimization strategy: during the latent denoising stage, a temporally weighted loss suppresses high-frequency noise, while in the latent reconstruction stage, pixel-level constraints restore low-frequency semantics, effectively recovering the input-latent mapping disrupted by defense mechanisms. By jointly optimizing a latent-image alignment loss, diffusion loss, and time-dependent weights, TS-LFO consistently bypasses state-of-the-art copyright protection schemes across diverse settings, achieving significantly higher attack success rates than baseline methods such as DiffPure, GrIDPure, and IMPRESS.

Abstract

With the growing concerns over copyright infringement in diffusion-based customization, adversarial attacks have emerged as a prominent defense strategy to prevent malicious content forgery in personalized image generation. However, current defenses typically introduce persistent perturbations in the latent space of Latent Diffusion Models (LDMs), which remain susceptible to adaptive bypasses by adversaries. In this paper, we introduce Two-Stage Latent Feature Optimization (TS-LFO), an efficient and effective copyright-stealing attack against protected diffusion-based customization. We begin by observing that existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model's ability to produce personalized outputs. To counteract this, TS-LFO restores the broken mapping through a two-stage optimization process. In the Latent Denoising Stage, we enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a Latent Diffusion Loss with timestep-dependent weights, effectively suppressing the high-frequency noise introduced by defenses. In the Latent Reconstruction Stage, we recover low-frequency semantic information using pixel-level constraints to refine the latent features. Extensive experiments show that TS-LFO consistently bypasses state-of-the-art (SOTA) copyright defenses and outperforms SOTA copyright attacks such as DiffPure, GrIDPure and IMPRESS across diverse settings.

Problem

• copyright protection
• diffusion models
• adversarial attacks
• latent space
• personalized image generation

Innovation

• Two-Stage Latent Feature Optimization
• Latent Diffusion Models
• Adversarial Attack
• Copyright Protection Bypass
• Latent-Image Alignment