Dual Attention Guided Defense Against Malicious Edits

2025-12-16

AI Summary

To address the security vulnerability of text-to-image diffusion models—where malicious text prompts can be exploited to generate harmful content during text-driven editing—this paper proposes a dual-attention-guided noise perturbation immunization method. Our approach simultaneously perturbs both the cross-attention mechanism and noise prediction across multiple denoising steps. It introduces a novel dynamic-threshold masking scheme that inversely modulates attention weights over text-relevant and text-irrelevant image regions, while maximizing semantic divergence between injected perturbations and the model’s predicted noise. By integrating cross-attention analysis, temporal noise perturbation optimization, dynamic mask generation, and attention reweighting, our method achieves fine-grained, semantics-aware defense. Evaluated on multiple benchmarks, it significantly reduces adversarial editing success rates—outperforming all prior state-of-the-art defenses—while preserving original editing fidelity and maintaining visual imperceptibility.

Technology Category

Computer Vision: Diffusion Models for VisionMachine Learning: Adversarial Learning & RobustnessNatural Language Processing: Safety and Robustness

Application Category

User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systemsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSecurity and Privacy: Large-scale security measurements

Abstract

Recent progress in text-to-image diffusion models has transformed image editing via text prompts, yet this also introduces significant ethical challenges from potential misuse in creating deceptive or harmful content. While current defenses seek to mitigate this risk by embedding imperceptible perturbations, their effectiveness is limited against malicious tampering. To address this issue, we propose a Dual Attention-Guided Noise Perturbation (DANP) immunization method that adds imperceptible perturbations to disrupt the model's semantic understanding and generation process. DANP functions over multiple timesteps to manipulate both cross-attention maps and the noise prediction process, using a dynamic threshold to generate masks that identify text-relevant and irrelevant regions. It then reduces attention in relevant areas while increasing it in irrelevant ones, thereby misguides the edit towards incorrect regions and preserves the intended targets. Additionally, our method maximizes the discrepancy between the injected noise and the model's predicted noise to further interfere with the generation. By targeting both attention and noise prediction mechanisms, DANP exhibits impressive immunity against malicious edits, and extensive experiments confirm that our method achieves state-of-the-art performance.

Problem

• Defends against malicious text-to-image diffusion model edits
• Uses dual attention and noise perturbation for immunization
• Preserves intended targets by misleading edits to incorrect regions

Innovation

• Dual Attention-Guided Noise Perturbation immunization method
• Manipulates cross-attention maps and noise prediction process
• Uses dynamic threshold masks to misguide edits