SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

2026-09-24

AI Summary

This study addresses the high communication overhead and accumulated consensus error caused by multi-step local updates in derivative-free decentralized federated learning. To this end, we propose SPADE-DFL, a method built upon a linearized primal-dual ADMM framework to optimize communication frequency. It innovatively permits the number of local updates to grow adaptively with computational capacity, thereby achieving client-level differential privacy, while rigorously decoupling data-dependent increments from graph correction terms to handle non-convex objectives. Experimental evaluations across four classification tasks demonstrate that SPADE-DFL attains superior average test accuracy compared to existing methods using fewer communication rounds. Furthermore, it preserves the theoretical convergence rate while significantly enhancing both communication efficiency and privacy guarantees.

Abstract

Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of $\mathcal{O}(T^{-1/3})$ using only $Θ(T^{2/3})$ communication rounds, where $T$ is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.

Problem

• Decentralized Federated Learning
• Communication Efficiency
• Derivative-Free Optimization
• Differential Privacy

Innovation

• Decentralized Federated Learning
• Derivative-Free Optimization
• Linearized ADMM
• Differential Privacy
• Communication Efficiency