Accurate and Resource-Efficient Federated Continual Learning

2026-06-09

AI Summary

This work addresses the challenges of federated continual learning under constraints on communication, computation, memory, and label availability by proposing the FedRAN framework. FedRAN introduces, for the first time, an analytical approach that replaces gradient transmission with compact statistics derived from random features and achieves linear communication complexity via truncated SVD compression of Gram matrices. At the server, QR-SVD enables two-level subspace fusion—across clients and across tasks—to obtain a closed-form ridge classifier, while prototype-based pseudo-labeling mitigates label scarcity. Evaluated on CIFAR-100, ImageNet-R, and VTAB benchmarks, FedRAN improves average accuracy by up to 4.8%, reduces communication overhead by 30.6–121.8×, and accelerates training by up to 190.3×; notably, with only 20% labeled data, pseudo-labeling further boosts accuracy by 6.61 percentage points.

Abstract

Federated continual learning (FCL) must learn from distributed task streams under limited resources, such as communication, computation, memory, and label availability. Existing FCL methods often rely on repeated local optimization, replay, and full supervision. Analytic alternatives avoid iterative training and replay, but using high-dimensional random features to improve accuracy requires a second-order feature statistic, the Gram matrix, which has a quadratic communication cost in the random feature size $M$. We propose FedRAN, a resource-aware analytic FCL framework that replaces gradient-based updates with compact random feature statistics. Each client transmits a truncated-SVD summary of its Gram matrix, reducing the dominant second-order upload from quadratic to linear in $M$ for fixed rank. The server performs a two-level QR-SVD subspace merge, spatially across clients and temporally across tasks, and solves a ridge classifier in closed form. FedRAN further supports label scarcity through prototype-based pseudo-labeling. Across CIFAR-100, ImageNet-R, and VTAB datasets, FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6-121.8$\times$ less per-client communication than optimization-based FCL, and is 190.3$\times$ faster on average than gradient-based baselines; with only 20% labels, pseudo-labeling improves average accuracy by up to 6.61 points. These results show that FedRAN enables accurate and resource-efficient FCL under communication, computation, and label constraints. The source code is available at https://github.com/JebacyrilArockiaraj/Fed-RAN-SSL.

Problem

• Federated Continual Learning
• Resource Efficiency
• Communication Constraints
• Label Scarcity
• Distributed Task Streams

Innovation

• Federated Continual Learning
• Analytic Learning
• Random Features
• Communication Efficiency
• Pseudo-labeling