🤖 AI Summary
To address instability in asynchronous federated learning and degradation of global model performance caused by Non-IID data and stale gradients, this paper proposes FedHist. Methodologically, FedHist introduces three key innovations: (1) a novel knowledge reclamation mechanism that caches and reuses historical global gradients to inject prior knowledge into local updates; (2) a multidimensional hindsight-weighted aggregation scheme that dynamically calibrates client contributions based on gradient freshness, magnitude, and directionality; and (3) an adaptive ℓ₂-norm gradient amplification strategy to mitigate bias induced by delayed gradients. Extensive experiments across multiple Non-IID benchmark datasets demonstrate that FedHist significantly accelerates convergence and improves test accuracy, while achieving superior stability and generalization compared to state-of-the-art asynchronous federated learning approaches.
📝 Abstract
Federated Learning (FL) allows several clients to cooperatively train machine learning models without disclosing the raw data. In practical applications, asynchronous FL (AFL) can address the straggler effect compared to synchronous FL. However, Non-IID data and stale models pose significant challenges to AFL, as they can diminish the practicality of the global model and even lead to training failures. In this work, we propose a novel AFL framework called Federated Historical Learning (FedHist), which effectively addresses the challenges posed by both Non-IID data and gradient staleness based on the concept of knowledge rumination. FedHist enhances the stability of local gradients by performing weighted fusion with historical global gradients cached on the server. Relying on hindsight, it assigns aggregation weights to each participant in a multi-dimensional manner during each communication round. To further enhance the efficiency and stability of the training process, we introduce an intelligent $ell_2$-norm amplification scheme, which dynamically regulates the learning progress based on the $ell_2$-norms of the submitted gradients. Extensive experiments indicate FedHist outperforms state-of-the-art methods in terms of convergence performance and test accuracy.
