FlashbackCL: Mitigating Temporal Forgetting in Federated Learning

2026-06-02

AI Summary

This work addresses temporal catastrophic forgetting in federated learning caused by time-varying client data distributions—a challenge overlooked by existing methods that typically assume static, stationary data. The paper formally defines a metric for temporal forgetting and introduces FlashbackCL, a novel framework that jointly mitigates spatial and temporal forgetting through three key components: time-decayed label counting, device-aware class-balanced reservoir sampling for replay, and server-side active coreset distillation coupled with knowledge distillation. Experiments demonstrate that FlashbackCL improves accuracy by 6.9%–10.0% over Flashback on non-stationary CIFAR-10 streams while reducing temporal forgetting by up to 68%. It also achieves a 3.5 percentage point gain on static CIFAR-100, confirming its effectiveness and generalization capability across both dynamic and stationary settings.

Abstract

Federated Learning (FL) of foundation and edge models increasingly targets deployments where client data distributions drift over time, yet existing forgetting-mitigation methods assume each client's distribution is stationary. Flashback, the strongest recent FL method against cross-client (spatial) forgetting, uses monotonically accumulating per-class label counts as a knowledge proxy; this proxy becomes miscalibrated under temporal distribution shift and anchors the global model to an outdated class balance. We formalise temporal forgetting in FL with a per-phase metric isolated from protocol-level fluctuations and propose Flashback Continual Learning (FlashbackCL), a drop-in extension of Flashback with (i) temporally-decayed label counts; (ii) a device-aware replay buffer with Class-Balanced Reservoir Sampling (CBRS); and (iii) server-side active coreset curation on the public distillation set. The results show that FlashbackCL achieves 6.9% to 10.0% relative improvement relative to Flashback, on CIFAR-10 with 50 clients and three controlled temporal shift modes, while simultaneously reducing temporal forgetting by up to 68%. A 5-variant ablation identifies CBRS replay as the critical component. FlashbackCL also improves Flashback by 3.5 points on stationary CIFAR-100, suggesting that class-balanced replay regularises spatial heterogeneity as well as temporal shift.

Problem

• Temporal Forgetting
• Federated Learning
• Distribution Shift
• Continual Learning
• Non-stationary Data

Innovation

• Temporal Forgetting
• Federated Learning
• Class-Balanced Reservoir Sampling
• Continual Learning
• Replay Buffer