Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models

2026-08-02

AI Summary

This work addresses the challenge that multimodal large language models lack fine-grained control over forgetting sensitive information at the attribute level, often failing to remove a target attribute while preserving other non-sensitive attributes of the same identity. To this end, the paper introduces the novel task of “attribute-level unlearning,” constructs a fine-grained evaluation benchmark encompassing long text, numerical values, and short text, and proposes Causal Localization with Retention-aware Projection (CLRP)—a training-free method. CLRP identifies critical causal layers via activation patching and applies retention-aware subspace projection to achieve precise, selective forgetting without model retraining. Experiments demonstrate that CLRP significantly improves the trade-off between forgetting efficacy and retention stability across multiple state-of-the-art multimodal large language models.

Abstract

Multimodal large language models (MLLMs) exhibit strong vision--language capabilities but may also memorize and disclose sensitive information. Machine unlearning seeks to remove designated knowledge without retraining from scratch while preserving general utility. Existing privacy-oriented benchmarks primarily adopt profile-level deletion, whereas practical requests are often finer grained: a model should forget a specified attribute while retaining non-sensitive information about the same identity. We therefore introduce attribute-level MLLM unlearning as a finer-grained task and construct a benchmark spanning long-text, numeric, and short-text targets, multiple forget ratios, and diverse question types. Our evaluation reveals that target and retained attributes share identity-specific and visual evidence, making selective forgetting susceptible to residual leakage or collateral degradation; accordingly, existing methods exhibit unstable forgetting--retention trade-offs in this setting. To address this challenge, we propose Causal Localization and Retain-Aware Projection (CLRP), a lightweight training-free framework. CLRP uses activation patching to identify the layer that causally mediates target-attribute disclosure, then applies a retain-aware projection that removes the target-attribute subspace while preserving same-identity evidence. Experiments across multiple widely used MLLMs with distinct architectures and parameter scales demonstrate the effectiveness of CLRP.

Problem

• attribute unlearning
• multimodal large language models
• fine-grained forgetting
• privacy preservation
• selective forgetting

Innovation

• attribute-level unlearning
• multimodal large language models
• machine unlearning
• causal localization
• retain-aware projection