From Mixing to Tearing: Graph Decomposition in Decentralized Optimization via Message Passing

2026-10-02

AI Summary

This study addresses the absence of a joint graph-structure design framework in existing decentralized optimization by proposing GATE. This method introduces a novel "hybrid-tearing" paradigm that decomposes the global problem into local subproblems via message passing, jointly designing constraint representations, dual variable blocks, and connected clusters to transcend conventional gossip communication limitations. Efficient collaborative computation is achieved through tree-recursive updates combined with a lightweight surrogate model (GATE-S). Theoretically, the algorithm is proven to attain a linear convergence rate under explicit dependencies on network topology and function regularity. Empirical evaluations further validate its significant advantages in reducing both communication and computational overhead.

Abstract

We study the minimization of sums of smooth strongly convex functions over undirected graphs, with each function held by one agent and communication restricted to neighbors in the graph. Existing decentralized methods, whether based on gossip or on routing over spanning trees, typically use the network to mix or aggregate information to enable {\it prescribed} local optimization updates. What this communication-centered viewpoint lacks is a general framework that uses graph structure to {\it jointly} design the optimization subproblems and the cooperative computation and communication through which agents solve them cooperatively. We develop such a framework from first principles, jointly designing the linear representation of agreement constraints, the blocks of the resulting dual variables (jointly optimized), and connected cluster of agents that cooperatively solve each block subproblem over the assigned subgraph. GATE (Graph-Tearing message passing) is a first instance of this framework: one variable per edge and tree blocks. At each iteration, agents update their assigned edge variables by minimizing the sum of the two endpoint cost-to-go messages and relaxing the result. The messages are updated through local minimizations following the tree recursion. To reduce per-iteration computational and communication costs, we develop GATE-S, a surrogate variant using tractable local models and lightweight message parametrizations. We establish linear convergence with a rate explicit in the interplay among function regularity, network topology, and the chosen partition, revealing the effects of graph decomposition. Numerical experiments are conducted to validate the theoretical results and evaluate the efficiency of our algorithms.

Problem

• Decentralized Optimization
• Graph Decomposition
• Message Passing
• Strongly Convex Functions
• Multi-Agent Systems

Innovation

• Decentralized Optimization
• Graph Decomposition
• Message Passing
• Graph Tearing
• Linear Convergence