Towards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers

2026-06-04

AI Summary

This work addresses the clustering challenge in fully decentralized federated learning arising from the coupling of network topology, data heterogeneity, and diverse local optimizers. To this end, the authors propose Serverless Semi-Decentralized Federated Learning (SSD-FL), a framework that performs lightweight device-to-device initialization for clustering without requiring a central server during training. SSD-FL employs a two-stage collaborative optimization process—within clusters (intra-cluster) and across clusters (inter-cluster). Its key innovation lies in introducing, for the first time in a serverless setting, an effective loss function that integrates graph-structured regularization with device-specific optimizers. An iterative clustering algorithm is further developed based on the Cheeger inequality and a heterogeneity score. Experiments demonstrate that SSD-FL consistently outperforms three representative decentralized baselines across various network topologies, datasets, and optimizer configurations, achieving notable improvements in convergence speed and communication efficiency.

Abstract

We investigate cluster formation, involving the number and composition of clusters, in decentralized federated learning (FL) with heterogeneous machine learning (ML) optimizers. While clustering in centralized FL has enabled scalability and resource savings, its value and development in fully decentralized environments have yet to be explored. Optimizing cluster formation in such environments is challenging, especially due to the complex coupling between network graph structures, local data heterogeneity, and different local ML model optimizers. To address these challenges, we propose serverless semi-decentralized FL (SSD-FL), a methodology requiring no persistent server infrastructure. In SSD-FL, cluster formation occurs via a lightweight, one-time device-to-device (D2D) initialization phase, after which actual ML model training (alongside consensus and convergence processes) is fully serverless. Functionally, SSD-FL segments global rounds into intra-cluster and inter-cluster regimes, ensuring global convergence and consensus through novel "effective loss functions" that integrate device-specific ML optimizers with network graph-based regularization. Next, SSD-FL leverages the consensus gap via the Cheeger inequality to develop an iterative clustering algorithm evaluated against our derived convergence and consensus bounds, which incorporate a unique scoring metric to quantify data and optimizer heterogeneity across devices. Finally, experimental evaluation against three categories of decentralized FL methodologies validate that SSD-FL improves both convergence speeds and communication efficiency across various network graphs, datasets, and local optimizer regimes.

Problem

• federated learning
• clustering
• heterogeneous optimizers
• decentralized learning
• serverless

Innovation

• serverless federated learning
• semi-decentralized
• heterogeneous optimizers
• cluster formation
• effective loss function