SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearning
This study addresses the challenge of privacy unlearning in vision-language models, where sensitive and retained information representations are inherently entangled. To this end, we propose a multimodal selective unlearning method based on attention value intervention. By treating attention values as pivotal intervention points, our approach suppresses personally identifiable information through attention value regularization. This mechanism is further integrated with frozen reference model matching, sequence-level negative cross-entropy, and retention supervision signals to jointly optimize precise unlearning and knowledge utility. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in multimodal settings, significantly mitigating utility degradation while effectively balancing privacy unlearning efficacy with the preservation of general knowledge.