Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression

2026-09-28

AI Summary

This study addresses the communication bottleneck in asynchronous federated learning by proposing two algorithms, AFL-BR and AFL-Com. AFL-BR leverages online mirror ascent and block-restart techniques to reduce the dependence of synchronous rounds on the number of clients from polynomial to logarithmic. AFL-Com is the first to integrate a logarithmic convergence rate with general compressors, employing bidirectional Top-k/Rand-k sparsification alongside an error feedback mechanism to optimize aggregation error control and substantially lower per-round communication costs. Experimental results demonstrate that these methods significantly reduce data transmission volume while preserving the convergence rate, thereby achieving efficient federated learning under low communication overhead.

Abstract

Agnostic federated learning (AFL) seeks a model that performs reliably across $m$ heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the number of synchronization rounds via faster convergence and the communication cost per round via compression. We first propose AFL-BR, which updates the dual weights over workers using online mirror ascent with KL divergence and blockwise restarts. It achieves an $O((\log m)^{1/4}T^{-1/8})$ stationarity rate after $T$ update rounds, reducing the $m$-dependence of the synchronization rounds required for convergence from polynomial to logarithmic order. Building on AFL-BR, we develop AFL-Com by applying bidirectional compression with error feedback (EF). Instead of compressing local gradients, workers apply EF to their dual-weighted gradients, enabling direct control of the aggregated compression error under time-varying weights. We then establish an $O((δ^{-1}+(\log m)^{1/4})T^{-1/8})$ stationarity rate for AFL-Com under general $δ$-approximate compressors and improve the $δ$-dependence from $δ^{-1}$ to $δ^{-1/2}$ for additive-and-idempotent compressors with shared randomness (SR). With suitable compression levels, AFL-Com retains the same convergence rate as AFL-BR at a lower per-round communication cost, yielding reductions in total communication complexity by factors of $(\log m)^{1/4}$ with Top-$k$ and $(\log m)^{1/2}$ with Rand-$k$ and SR. Experiments validate the improved synchronization and communication efficiency of our methods.

Problem

• Agnostic Federated Learning
• Communication Efficiency
• Heterogeneous Workers
• Compression

Innovation

• Agnostic Federated Learning
• Communication Compression
• Error Feedback
• Online Mirror Ascent
• Convergence Rate