Technology Category
Machine Learning: Distributed Machine Learning & Federated LearningMultiagent Systems: Adversarial AgentsSearch and Optimization: Learning to Search
FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures
🤖 AI Summary
In heterogeneous federated learning, disparities in model architectures (e.g., CNNs, RNNs, Transformers) and device computational capabilities impede effective aggregation, leading to slow convergence and poor generalization. To address this, we propose a unified aggregation framework with dynamic architecture adaptation. Our key contributions are: (1) a novel differentiable architecture projection (DAP) mechanism that enables parameter-level alignment across heterogeneous architectures without requiring client-side model standardization or modification; and (2) a synergistic aggregation strategy integrating gradient-aware weight distillation and capability-aware elastic weighting, enhancing both collaboration efficiency and robustness. Extensive experiments on CIFAR-10, CIFAR-100, and LEAF datasets demonstrate that our method achieves up to 23.30% higher accuracy than FlexiFed, reduces communication overhead by 37%, and accelerates convergence by 2.1×.
Technology Category
Application Category
📝 Abstract
Traditional Federated Learning (FL) faces significant challenges in terms of efficiency and accuracy, particularly in heterogeneous environments where clients employ diverse model architectures and have varying computational resources. Such heterogeneity complicates the aggregation process, leading to performance bottlenecks and reduced model generalizability. To address these issues, we propose FedADP, a federated learning framework designed to adapt to client heterogeneity by dynamically adjusting model architectures during aggregation. FedADP enables effective collaboration among clients with differing capabilities, maximizing resource utilization and ensuring model quality. Our experimental results demonstrate that FedADP significantly outperforms existing methods, such as FlexiFed, achieving an accuracy improvement of up to 23.30%, thereby enhancing model adaptability and training efficiency in heterogeneous real-world settings.
Problem
Research questions and friction points this paper is trying to address.
Addresses efficiency and accuracy challenges in Federated Learning with heterogeneous model architectures
Solves aggregation issues caused by diverse client models and computational resources
Improves model adaptability and training efficiency in real-world heterogeneous settings
Innovation
Methods, ideas, or system contributions that make the work stand out.
Dynamic model architecture adjustment during aggregation
Effective collaboration among heterogeneous clients
Improved accuracy and resource utilization
🔎 Similar Papers
No similar papers found.
