Uni-AdaVD: Universal Concept Erasure for Visual Generation via Orthogonal Value Decomposition

2026-07-15

AI Summary

Existing vision generative models often learn undesirable concepts during pretraining, yet current concept-erasure methods are typically architecture-specific and struggle to simultaneously remove target concepts while preserving non-target generative priors. This work proposes a general inference-time framework for concept erasure by treating the multimodal attention value space as a unified intervention region. Leveraging encoder-aware target representation construction, orthogonal value decomposition, and adaptive semantic direction suppression, the method precisely eliminates target concepts without modifying model weights. It achieves, for the first time, universal erasure across diverse architectures—including U-Net, DiT, autoregressive image, and text-to-video models—and supports both single and multiple concept removal. Extensive experiments demonstrate significant performance gains over existing approaches across mainstream architectures, striking an effective balance between safety and generation quality.

Abstract

Visual generative models inevitably absorb undesirable concepts from uncurated pretraining data, making concept erasure essential for safe deployment. Existing erasure methods, however, are often architecture-specific and struggle to remove target concepts while preserving non-target content and generative priors. We present Uni-AdaVD, a universal inference-time concept erasure framework for visual generation. Uni-AdaVD treats the value space of multimodal attention as a unified intervention space and introduces encoder-aware target representation construction to localize target semantics across heterogeneous text encoders. It further combines orthogonal value decomposition with an adaptive erasing shift to suppress target semantic directions without updating the original model weights. Extensive experiments on U-Net-, DiT-, and autoregressive image generators, as well as text-to-video models, demonstrate strong performance on single- and multi-concept erasure while preserving non-target priors. These results suggest that Uni-AdaVD provides an efficient and adaptable safety mechanism for modern visual generative models. Our code is available at https://github.com/QifanZhou/Uni-AdaVD.

Problem

• concept erasure
• visual generation
• undesirable concepts
• generative priors
• multimodal attention

Innovation

• concept erasure
• orthogonal value decomposition
• universal visual generation
• inference-time intervention
• multimodal attention