VOID: Defeating Unauthorized Mimicry in Latent Diffusion Models

2026-06-10

AI Summary

This work addresses the misuse of latent diffusion models (LDMs) for unauthorized identity imitation by proposing a novel defense mechanism that manipulates the intrinsic randomness of the diffusion process. By injecting semantic-level perturbations into the latent space, the method amplifies encoding errors and suppresses target-guidance signals, thereby disrupting identity reproduction while preserving visual fidelity. Unlike conventional approaches relying on imperceptible input perturbations—which are often mitigated by the model’s generative priors—this strategy operates directly within the diffusion dynamics, offering the first internal defense against identity theft. Comprehensive evaluations across five datasets, ten imitation attacks, and twenty-four baseline defenses demonstrate its efficacy: the Fréchet Inception Distance (FID) increases from 113 to 365, reflecting a 223% improvement over the strongest existing baseline.

Abstract

While Latent Diffusion Models (LDMs) have revolutionized visual synthesis, they are increasingly exploited for unauthorized mimicry of individuals. Existing defenses inject deceptive perturbations to steer the generated images toward irrelevant targets. However, this approach hinges on an ungrounded assumption: subtle perturbations can maintain their deceptive efficacy throughout an LDM's extensive generation process. In reality, the model's innate restoration mechanism will remove such perturbations and cause individual identities to re-emerge in the images generated. We propose VOID, a defense framework that overcomes this conundrum by manipulating an LDM's intrinsic stochasticity. VOID perturbs the diffusion pipeline in two novel ways: 1) amplifying the latent encoding errors to shatter an image's semantic structure, and 2) counteracting the target guidance signals to suppress the model's restoration capabilities. This results in a semantic corruption that thwarts any unauthorized mimicry. Notably, the security gain does not come at the price of visual utility, as VOID simultaneously manages to confine perturbations to human-imperceptible regions of protected images. Our comprehensive evaluation of 24 state-of-the-art defenses against 10 mimicry attacks on 5 datasets demonstrates VOID's unprecedented protection power: it increases the average Frechet Inception Distance (FID) from 113 to 365, a 223% improvement over the strongest defense to date.

Problem

• unauthorized mimicry
• Latent Diffusion Models
• identity protection
• image generation abuse

Innovation

• Latent Diffusion Models
• Unauthorized Mimicry
• Stochasticity Manipulation
• Semantic Corruption
• Adversarial Defense