From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges

2025-03-10

AI Summary

This work identifies a critical oversight in federated learning research: while existing studies emphasize network topology, they neglect the fundamental distinction between centralized (CFL) and decentralized federated learning (DFL)—namely, their training protocols (decoupled aggregation vs. joint optimization). To address this, we propose the first protocol-centric taxonomy for CFL/DFL. We systematically expose a long-standing research gap concerning distributed optimization methods in DFL and establish a theoretical triadic trade-off model among privacy, robustness, and model utility. Leveraging protocol-driven analysis, distributed optimization theory, and meta-review methodology, we develop a unified analytical framework for CFL and DFL. Our framework rigorously characterizes how distributed optimization fundamentally enables DFL, thereby providing a principled foundation and design guidance for next-generation federated learning systems that are secure, scalable, and adversarially robust.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated Learning (FL) enables collaborative learning without directly sharing individual's raw data. FL can be implemented in either a centralized (server-based) or decentralized (peer-to-peer) manner. In this survey, we present a novel perspective: the fundamental difference between centralized FL (CFL) and decentralized FL (DFL) is not merely the network topology, but the underlying training protocol: separate aggregation vs. joint optimization. We argue that this distinction in protocol leads to significant differences in model utility, privacy preservation, and robustness to attacks. We systematically review and categorize existing works in both CFL and DFL according to the type of protocol they employ. This taxonomy provides deeper insights into prior research and clarifies how various approaches relate or differ. Through our analysis, we identify key gaps in the literature. In particular, we observe a surprising lack of exploration of DFL approaches based on distributed optimization methods, despite their potential advantages. We highlight this under-explored direction and call for more research on leveraging distributed optimization for federated learning. Overall, this work offers a comprehensive overview from centralized to decentralized FL, sheds new light on the core distinctions between approaches, and outlines open challenges and future directions for the field.

Problem

• Explores differences between centralized and decentralized Federated Learning protocols.
• Analyzes impact of protocols on model utility, privacy, and attack robustness.
• Identifies lack of research on decentralized FL using distributed optimization methods.

Innovation

• Decentralized Federated Learning uses peer-to-peer networks.
• Distinguishes centralized vs decentralized by training protocols.
• Highlights distributed optimization for enhanced privacy and robustness.