Problem
Research questions and friction points this paper is trying to address.
Concept Erasure
Diffusion Models
Semantic Prototypes
Gaussian Mixture Model
Low-Rank Adaptation
Gaussian Core LoRA: Distribution-Aware Dynamic Adaptation for Broad Concept Erasure
🤖 AI Summary
本文针对文本到图像生成模型中广泛概念擦除问题，提出了一种基于高斯混合模型的分布感知低秩适应方法Gaussian Core LoRA，以实现更精确的概念抑制。
📝 Abstract
Concept erasure aims to suppress unsafe, privacy-sensitive, or undesirable generations in text-to-image diffusion models while preserving benign semantics, visual quality, and deployment efficiency. Existing adapter-based methods, such as Low-Rank Adaptation (LoRA), typically freeze the diffusion backbone and learn lightweight parameter updates to steer generation away from target semantics. However, these methods usually assign a static semantic erasure direction to each target concept. This assumption is overly coarse for broad and complex target concepts, since a concept often contains multiple latent semantic prototypes involving different objects, scenes, or relations, and requires different local erasure directions. A single LoRA update averages these heterogeneous erasure demands, leading to under-erasure on difficult prototypes and over-editing of nearby benign semantics. To address this limitation, we propose Gaussian Core LoRA, a distribution-aware low-rank adaptation framework. It fits a Gaussian mixture model in the prompt feature space to estimate latent semantic prototypes within the target concept. During inference, each input prompt is projected into this feature space to compute its Gaussian posterior responsibilities, which condition the core generator to produce a prompt-specific, norm-bounded residual reconfiguration of the shared LoRA rank space. This enables prototype-adaptive erasure with a single lightweight adapter. Compared with the strongest baseline on each metric, Gaussian Core LoRA reduces average Attack Success Rate (ASR) by 7.95%, lowers COCO Fr'echet Inception Distance (FID) by 14.72%, and improves CLIP Score by 4.98%. Further experiments show robustness to adversarial prompts, scalability to multi-identity and multi-style erasure, and compatibility with SDXL and FLUX.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Gaussian Mixture Model
Distribution-Aware
Prototype-Adaptive Erasure
Low-Rank Adaptation (LoRA)
Semantic Prototypes
🔎 Similar Papers
No similar papers found.
