Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning

2025-06-11

AI Summary

To address the training efficiency bottleneck in decentralized federated learning under the model circulation paradigm, this paper proposes a load-aware joint computation-communication scheduling mechanism to minimize end-to-end global training latency. Methodologically, it decomposes the global scheduling problem into distributed node-level subproblems; introduces variance-constrained regularization to ensure balanced utilization of local model updates under non-IID data distributions; and establishes a fine-grained load model integrating heterogeneous computational capabilities and communication bandwidths. Experimental evaluation on MNIST and CIFAR-10 demonstrates that the proposed approach reduces total training time by 37%, accelerates convergence by 2.1×, and decreases communication overhead by 29% compared to baseline methods.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Planning, Routing, and Scheduling: Planning/Scheduling and Learning

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

This paper proposes Load-aware Tram-FL, an extension of Tram-FL that introduces a training scheduling mechanism to minimize total training time in decentralized federated learning by accounting for both computational and communication loads. The scheduling problem is formulated as a global optimization task, which-though intractable in its original form-is made solvable by decomposing it into node-wise subproblems. To promote balanced data utilization under non-IID distributions, a variance constraint is introduced, while the overall training latency, including both computation and communication costs, is minimized through the objective function. Simulation results on MNIST and CIFAR-10 demonstrate that Load-aware Tram-FL significantly reduces training time and accelerates convergence compared to baseline methods.

Problem

• Minimize total training time in decentralized federated learning
• Balance data utilization under non-IID distributions
• Reduce training latency and accelerate convergence

Innovation

• Load-aware training scheduling minimizes total training time
• Decomposes global optimization into node-wise subproblems
• Introduces variance constraint for balanced non-IID data utilization