Scalar Communication via Random Direction Refreshing for Distributed Optimization

2026-10-05

AI Summary

This study addresses the high communication overhead in high-dimensional distributed optimization and the complexity of error compensation in existing compression methods by proposing a continuous-time cooperative optimization framework based on scalar communication. The approach leverages isotropic distributions, such as Rademacher sampling, to generate shared random directions, projecting agent states and transmitting only a single real-valued inner product to overcome dimensionality limitations. Furthermore, spurious equilibria are eliminated through a rank-one surrogate combined with a high-frequency refreshment mechanism. Theoretically, the existence of a unique consensus equilibrium is established under strongly convex Lipschitz conditions, achieving residual-free exponential convergence. Simulations validate the variance advantages of different directional distributions and demonstrate the algorithm's efficient convergence performance.

Abstract

Distributed optimization over networks requires agents to repeatedly exchange decision variables with their neighbors. When the decision dimension $d$ is large, these exchanges dominate the communication cost, which is critical for bandwidth-constrained agents. Existing remedies quantize or sparsify the exchanged vectors, yet each message still scales with $d$ and the compression error must be compensated by additional states. To address this limitation, we propose a scalar-communication mechanism in which every neighbor message carries a single real number regardless of $d$. Agents regenerate a common random direction from a shared seed, transmit only the inner product of their state with that direction, and act on the resulting rank-one surrogate of their neighbors'states while retaining full local gradients. We develop and analyze the mechanism for an existing continuous-time distributed optimization algorithm. For strongly convex local costs with Lipschitz gradients, we show that the optimizer remains the unique consensus equilibrium, that a fixed direction admits spurious equilibria, and that refreshing the direction at a sufficiently high rate yields exponential mean-square and almost-sure convergence with constant gains and no residual error. The framework admits any isotropic fixed-norm direction distribution, including Rademacher, scaled-coordinate, and sphere-normalized Gaussian directions; all three attain lower fresh-encoding variance than unnormalized Gaussian directions. The effects of the direction distribution and the refresh interval are illustrated in~simulations.

Problem

• Distributed Optimization
• Communication Efficiency
• Scalar Communication
• Bandwidth-constrained Networks

Innovation

• Distributed Optimization
• Scalar Communication
• Random Direction Refreshing
• Rank-one Surrogate
• Communication Efficiency