Lethe: How Hard Is It to Forget? A Benchmark for Federated Unlearning in Medical Imaging

2026-08-02

AI Summary

This work addresses the need for machine unlearning in medical federated learning to support the "right to be forgotten," noting that existing unlearning methods are primarily designed for natural images and poorly generalize to medical imaging. To bridge this gap, the authors propose Lethe, the first unified evaluation benchmark for federated unlearning tailored to medical images. They systematically assess 12 unlearning methods across eight diverse tasks—including classification, segmentation, denoising, cross-modal synthesis, and visual-language question answering—and three unlearning granularities, using full retraining as the gold standard to evaluate utility preservation, privacy guarantees, and computational cost. Their findings reveal that unlearning difficulty, rather than algorithmic design, predominantly dictates performance; due to strong cross-institutional generalization, forgetting a single client often minimally impacts model utility, with residual membership information—not utility loss—posing the primary privacy risk; and distinctions among current methods emerge only in complex unlearning scenarios.

Abstract

Federated learning enables medical-imaging models to be trained across hospitals, and privacy law, most explicitly the GDPR ``right to be forgotten'', turns removing a hospital's, a class's, or a patient's influence from such a model into a federated unlearning problem. This need is most acute in medicine, where patients withdraw consent and hospitals leave collaborations. Yet nearly all unlearning evidence comes from natural images, whose heterogeneity and task structure differ sharply from clinical data, so it is unclear whether existing methods transfer, and no shared protocol covers clinical data. We present Lethe, a benchmark for federated unlearning in medical imaging. It evaluates twelve methods across eight task families, from classification and segmentation to denoising, cross-modality synthesis, and vision-language question answering, at three forgetting granularities and against a retrained gold standard on utility, privacy, and cost. The central result is that what separates methods is the difficulty of the forgetting request, not the method itself. The easy removals that dominate the literature leave the methods that preserve utility indistinguishable, while only hard ones separate them. More striking, on the many medical tasks that generalize across sites, forgetting a client barely changes task performance, leaving residual membership as the signal that must be erased.

Problem

• federated unlearning
• medical imaging
• right to be forgotten
• privacy
• federated learning

Innovation

• federated unlearning
• medical imaging
• forgetting benchmark
• membership privacy
• cross-site generalization