MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs

2026-06-10

AI Summary

This work addresses the challenge of lifelong unlearning in multimodal large language models, where repeated data deletion requests often lead to significant performance degradation and difficulty in maintaining cross-modal alignment. To this end, the authors first systematically characterize the unique constraints of multimodal unlearning, emphasizing the necessity of preserving alignment across modalities during forgetting. They propose LUMoE, a novel approach based on a Mixture-of-Experts (MoE) architecture, and introduce MLUBench—the first large-scale benchmark for multimodal lifelong unlearning—covering 127 entities across nine categories. Experiments demonstrate that existing methods suffer from substantial cumulative performance decline on MLUBench, whereas LUMoE effectively executes continual unlearning while significantly mitigating performance loss and preserving cross-modal alignment.

Abstract

Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of MLLM Lifelong Unlearning. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a large-scale and comprehensive benchmark featuring 127 entities across 9 classes under lifelong unlearning requests. We perform extensive experiments using MLUBench and reveal that existing unlearning methods suffer from severe, cumulative degradation. More critically, we further identify the unique challenge of this problem: unlike in unimodal models, MLLM lifelong unlearning is constrained by the need to preserve multimodal alignment. Continually unlearning from one modality could degrade the entire model. To alleviate this challenge, we propose LUMoE, an effective method. Experiments demonstrate that LUMoE significantly mitigates the degradation problem faced by baselines. The source code and the MLUBench dataset are open-sourced in https://github.com/lihe-maxsize/Lifelong_Unlearning_main.

Problem

• Lifelong Unlearning
• Multimodal Large Language Models
• Data Unlearning
• Multimodal Alignment
• MLLMs

Innovation

• Lifelong Unlearning
• Multimodal Large Language Models
• Unlearning Benchmark
• Multimodal Alignment
• LUMoE