🤖 AI Summary
To address excessive communication overhead in asynchronous ADMM for distributed optimization and federated learning, this paper proposes Quantized Asynchronous ADMM (Q-Async-ADMM), which integrates coarse low-bit quantization into the variable exchange phase. The method preserves both asynchrony and convergence guarantees while drastically reducing inter-node data transmission. We establish theoretical convergence under nonconvex and nonsmooth objectives. Empirical evaluation across multiple federated learning and distributed training tasks demonstrates 75–90% reduction in communication volume, with convergence speed and accuracy matching full-precision baselines; the approach further exhibits strong scalability to complex models such as deep neural networks. To our knowledge, this is the first work to systematically incorporate coarse quantization into the asynchronous ADMM framework, yielding an efficient and robust distributed optimization paradigm tailored for communication-constrained environments.
📝 Abstract
In distributed optimization and federated learning, asynchronous alternating direction method of multipliers (ADMM) serves as an attractive option for large-scale optimization, data privacy, straggler nodes and variety of objective functions. However, communication costs can become a major bottleneck when the nodes have limited communication budgets or when the data to be communicated is prohibitively large. In this work, we propose introducing coarse quantization to the data to be exchanged in aynchronous ADMM so as to reduce communication overhead for large-scale federated learning and distributed optimization applications. We experimentally verify the convergence of the proposed method for several distributed learning tasks, including neural networks.