Problem
Research questions and friction points this paper is trying to address.
Private Decentralized Learning
Sampling Noise
Privacy Noise
Decentralized Bias
Heterogeneous Data
Private Decentralized Optimization with Noise Reduction and Bias Correction
🤖 AI Summary
本文针对私有去中心化学习中的噪声和偏差问题，提出了一种通过递归估计和精确扩散来减少误差和纠正偏差的新方法PRDO。
📝 Abstract
Private decentralized learning is affected by sampling noise, privacy noise, and decentralized bias under heterogeneous data. We propose Private Recursive Decentralized Optimization (PRDO). PRDO uses recursive estimation with same-batch gradient differences to reduce estimation errors caused by sampling and privacy noise, while its Exact Diffusion component corrects decentralized bias arising from data heterogeneity. Our analysis establishes a nonconvex convergence bound without assuming uniformly bounded data heterogeneity across nodes. It further gives a sufficient condition under which recursive gradient differences yield strictly lower query sensitivity than private Exact Diffusion, together with an example that rigorously satisfies this condition. Experiments show improved accuracy over the evaluated baselines.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Private Recursive Decentralized Optimization
Recursive Estimation
Exact Diffusion
Data Heterogeneity
Query Sensitivity
🔎 Similar Papers
No similar papers found.
