Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning

2026-10-04

AI Summary

This study addresses the degradation of information propagation and consensus dynamics caused by client scaling in decentralized learning. It systematically investigates the coupled effects of data distribution, topological mixing, and communication capacity, proposing a local heuristic topology evolution mechanism (LFHE) based on “friend-of-a-friend” discovery. Through spectral gap analysis and CIFAR-10 experiments comparing ring and static random graph topologies, this work reveals that network scalability is jointly governed by learning and communication dynamics. The findings demonstrate that fixing local data volume can mitigate the population penalty. Furthermore, LFHE significantly outperforms baseline methods in model accuracy and consensus efficiency, albeit at the cost of higher transmission overhead.

Abstract

Scaling decentralized learning changes not only the number of clients $N$, but also the dynamics of information propagation and consensus. We argue that the effect of increasing $N$ cannot be understood in isolation, because data allocation, topology-dependent mixing, and communication capacity may change simultaneously. We study these coupled effects on CIFAR-10 with $N\in\{10,50,100,200\}$, comparing a degree-two Ring, a Static Random graph, and Local-First Heuristic Evolution (LFHE), a locally adaptive topology process based on friend-of-friend discovery. The Ring provides an analytically transparent failure mode: its Metropolis spectral gap decays as $\Theta(N^{-2})$, implying progressively slower contraction of model disagreement as the population grows. Experiments show that holding the nominal local dataset size fixed substantially reduces the apparent population penalty observed when a fixed total dataset is divided among more clients. The remaining degradation depends strongly on communication structure: Ring enters a high-disagreement regime, whereas Static Random and LFHE remain close to consensus. Increasing LFHE's degree threshold further improves accuracy and consensus, but at a substantially higher model-transmission cost. These results show that decentralized scaling is governed by coupled learning and communication dynamics, rather than by the number of clients alone.

Problem

• Decentralized Learning
• Population Scaling
• Data Dilution
• Topology Evolution
• Consensus Dynamics

Innovation

• Decentralized Learning
• Topology Evolution
• Spectral Gap
• Population Scaling
• Local-First Heuristic Evolution