Dimension-Free Decentralized Nonsmooth Nonconvex Stochastic Optimization

2026-10-05

AI Summary

This study addresses the efficiency bottleneck in high-dimensional decentralized non-smooth non-convex stochastic optimization, where existing algorithms suffer from polynomial dependence on dimensionality. To overcome this limitation, this work proposes a novel algorithm leveraging decentralized online convex optimization (D-OCO) and an online-to-non-convex conversion technique. The core innovation lies in revealing the intrinsic connection between consensus error and dimensional dependence, demonstrating that merely logarithmic additional communication suffices to eliminate polynomial dimensional terms, thereby efficiently reducing the original problem to a D-OCO formulation. Consequently, the proposed method achieves an $\mathcal{O}(\delta^{-1}\varepsilon^{-3})$ sample complexity and an $\widetilde{\mathcal{O}}(\gamma^{-1/2}\delta^{-1}\varepsilon^{-3})$ communication complexity, significantly outperforming existing approaches.

Abstract

We investigate decentralized nonsmooth nonconvex stochastic optimization over a network of $n$ nodes, with the goal of finding an $(\delta,\epsilon)$-Goldstein stationary point. The best existing algorithm achieves $O(\delta^{-1}(\epsilon^{-3}+d\epsilon^{-1}))$ sample complexity and $\widetilde{O}(\gamma^{-1/2}\delta^{-1}(\epsilon^{-3}+d\epsilon^{-1}))$ communication complexity, where $d$ is the problem dimension and $\gamma$ is the spectral gap of the communication matrix. However, the polynomial dependence on $d$ can be a major bottleneck in high-dimensional regimes. In this paper, we propose a novel algorithm that achieves $O(\delta^{-1}\epsilon^{-3})$ sample complexity and $\widetilde{O}(\gamma^{-1/2}\delta^{-1}\epsilon^{-3})$ communication complexity. The primary technique is an elegant decentralized online-to-nonconvex conversion that reduces the original problem to a decentralized online convex optimization (D-OCO) problem. A key property of our conversion is that its consensus requirements can be inherited directly from the consensus of the underlying D-OCO decisions. In particular, this property enables us to establish an explicit connection between the dimension dependence and the consensus error, which in turn shows that the polynomial dependence on $d$ can be removed with only logarithmic additional communication.

Problem

• decentralized optimization
• nonsmooth nonconvex
• stochastic optimization
• dimension-free
• Goldstein stationary point

Innovation

• Decentralized Optimization
• Nonsmooth Nonconvex Stochastic Optimization
• Dimension-Free Complexity
• Online-to-Nonconvex Conversion
• Goldstein Stationary Point