Fundamental Limits of Distributed Computing for Linearly Separable Functions

2025-09-27

AI Summary

This paper addresses the joint communication-computation optimization problem for linearly separable functions in distributed computing: a master node holds $K$ datasets, and $N$ servers collaboratively compute $L$ linear functions requested by users, with the requirement that any $L$-subset of these functions must be recoverable without error. We propose a unified design of task assignment and coded transmission based on nullspace properties, revealing for the first time the duality between nullspace structure and sparse matrix factorization. The minimum communication overhead is exactly characterized as the covering number from combinatorial covering design. Without assuming data sub-grouping, our approach employs linear coding and information-theoretic dual bounds to establish fundamental trade-offs between communication and computation for arbitrary $K$ and $L$. The proposed scheme achieves exact optimality in general cases; otherwise, it attains within a factor of three of the optimal communication cost.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search
• Machine Learning: Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Applications of cryptography

Abstract

This work addresses the problem of distributed computation of linearly separable functions, where a master node with access to $K$ datasets, employs $N$ servers to compute $L$ user-requested functions, each defined over the datasets. Servers are instructed to compute subfunctions of the datasets and must communicate computed outputs to the user, who reconstructs the requested outputs. The central challenge is to reduce the per-server computational load and the communication cost from servers to the user, while ensuring recovery for any possible set of $L$ demanded functions. We here establish the fundamental communication-computation tradeoffs for arbitrary $K$ and $L$, through novel task-assignment and communication strategies that, under the linear-encoding and no-subpacketization assumptions, are proven to be either exactly optimal or within a factor of three from the optimum. In contrast to prior approaches that relied on fixed assignments of tasks -- either disjoint or cyclic assignments -- our key innovation is a nullspace-based design that jointly governs task assignment and server transmissions, ensuring exact decodability for all demands, and attaining optimality over all assignment and delivery methods. To prove this optimality, we here uncover a duality between nullspaces and sparse matrix factorizations, enabling us to recast the distributed computing problem as an equivalent factorization task and derive a sharp information-theoretic converse bound. Building on this, we establish an additional converse that, for the first time, links the communication cost to the covering number from the theory of general covering designs.

Problem

• Minimizing computational load and communication cost in distributed computing
• Establishing fundamental tradeoffs for arbitrary dataset and function parameters
• Developing optimal task assignment and communication strategies for exact decodability

Innovation

• Nullspace-based design for task assignment
• Duality between nullspaces and sparse matrix factorizations
• Communication cost linked to covering number theory