Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks

2025-01-20

AI Summary

To address high communication overhead, slow convergence, and sensitivity to device heterogeneity and channel instability in federated learning (FL) over heterogeneous wireless edge networks, this paper proposes FedQVR—a novel FL framework featuring the first variance-reduction mechanism under quantization constraints. FedQVR supports heterogeneous local updates and ultra-low-bit (1–2 bit) model transmission, while theoretically guaranteeing an $O(1/T)$ convergence rate. Furthermore, we design FedQVR-E, a resource-aware joint bandwidth and quantization-bit allocation algorithm tailored to non-ideal channel conditions. Extensive experiments demonstrate that FedQVR reduces communication volume by 3.2× compared to state-of-the-art efficient FL methods, improves model accuracy by 2.7%, and significantly enhances energy efficiency and robustness against device and channel fluctuations.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hindered by the high communication overhead caused by frequent and costly server-device synchronization. Notably, most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance resulting from the prevalent issue of device heterogeneity. This variance severely decelerates algorithm convergence, increasing communication overhead and making it more challenging to achieve a well-performed model. In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices. Comprehensive theoretical analysis justifies that FedQVR is inherently resilient to device heterogeneity and has a comparable convergence rate even with a small number of quantization bits, yielding significant communication savings. Besides, considering non-ideal wireless channels, we propose FedQVR-E which enhances the convergence of FedQVR by performing joint allocation of bandwidth and quantization bits across devices under constrained transmission delays. Extensive experimental results are also presented to demonstrate the superior performance of the proposed algorithms over their counterparts in terms of both communication efficiency and application performance.

Problem

• Federated Learning
• Communication Efficiency
• Device Performance Variability

Innovation

• FedQVR
• Energy Efficiency
• Wireless Network Optimization