Asynchronous Personalized Federated Learning through Global Memorization

2025-03-01

AI Summary

To address statistical and system heterogeneity in federated learning—arising from non-IID data distributions, frequent client dropouts (especially monopolistic clients), and stringent privacy regulations prohibiting raw-data uploads—this paper proposes an asynchronous personalized federated learning framework tailored for IoT scenarios. The method introduces three key innovations: (1) a server-side data-free semantic generator enabling zero-shot synthesis of unseen-class samples to mitigate data scarcity caused by client dropouts; (2) a global memory-augmented semantic generation mechanism coupled with a decoupled model interpolation strategy; and (3) the first integration of zero-shot learning into federated training to robustly handle monopolistic client failures. Crucially, knowledge distillation and personalized optimization are achieved without accessing raw client data. Evaluated on multiple non-IID benchmarks, the approach achieves average accuracy gains of 5.2–9.8%, maintains strong robustness under 60% client dropout rates, and accelerates convergence by 40%.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The proliferation of Internet of Things devices and advances in communication technology have unleashed an explosion of personal data, amplifying privacy concerns amid stringent regulations like GDPR and CCPA. Federated Learning offers a privacy preserving solution by enabling collaborative model training across decentralized devices without centralizing sensitive data. However, statistical heterogeneity from non-independent and identically distributed datasets and system heterogeneity due to client dropouts particularly those with monopolistic classes severely degrade the global model's performance. To address these challenges, we propose the Asynchronous Personalized Federated Learning framework, which empowers clients to develop personalized models using a server side semantic generator. This generator, trained via data free knowledge transfer under global model supervision, enhances client data diversity by producing both seen and unseen samples, the latter enabled by Zero-Shot Learning to mitigate dropout-induced data loss. To counter the risks of synthetic data impairing training, we introduce a decoupled model interpolation method, ensuring robust personalization. Extensive experiments demonstrate that AP FL significantly outperforms state of the art FL methods in tackling non-IID distributions and client dropouts, achieving superior accuracy and resilience across diverse real-world scenarios.

Problem

• Addresses privacy concerns in federated learning with GDPR and CCPA compliance.
• Mitigates performance degradation from non-IID data and client dropouts.
• Enhances model personalization and resilience using synthetic data generation.

Innovation

• Asynchronous Personalized Federated Learning framework
• Server-side semantic generator for data diversity
• Decoupled model interpolation for robust personalization