Locally Differentially Private Online Federated Learning With Correlated Noise

2024-11-27

arXiv.org

AI Summary

Online federated learning over streaming non-IID data poses challenges in simultaneously ensuring privacy, adaptability to concept drift, and convergence under local updates. Method: This paper proposes the first dynamic optimization framework for online federated learning satisfying local differential privacy (LDP). It introduces a time-correlated noise mechanism to improve the utility–privacy trade-off under $(epsilon,delta)$-LDP and develops novel perturbed iterate analysis tools to derive the first quantifiable dynamic regret bound—explicitly parameterized by environmental change intensity—under LDP. Contribution/Results: The framework provably suppresses drift error induced by local updates on non-convex losses, and its dynamic regret scales controllably with the environment’s variation rate. Empirical evaluation demonstrates significantly higher utility than independent-noise baselines, establishing new state-of-the-art for privacy-preserving adaptive federated learning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

We introduce a locally differentially private (LDP) algorithm for online federated learning that employs temporally correlated noise to improve utility while preserving privacy. To address challenges posed by the correlated noise and local updates with streaming non-IID data, we develop a perturbed iterate analysis that controls the impact of the noise on the utility. Moreover, we demonstrate how the drift errors from local updates can be effectively managed for several classes of nonconvex loss functions. Subject to an $(epsilon,delta)$-LDP budget, we establish a dynamic regret bound that quantifies the impact of key parameters and the intensity of changes in the dynamic environment on the learning performance. Numerical experiments confirm the efficacy of the proposed algorithm.

Problem

• Privacy Preservation
• Online Collaborative Learning
• Environmental Impact on Learning

Innovation

• Privacy Protection
• Adaptive Noise
• Online Collaborative Learning