Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning

2026-03-16

AI Summary

This work addresses nonconvex constrained optimization problems lacking global Lipschitz smoothness by introducing a novel regularity assumption termed the Dual Kernel Condition (DKC). Within the framework of relative smoothness, DKC establishes Lipschitz continuity of gradients in the dual space. Leveraging DKC together with mirror maps, Bregman distances, and stochastic reshuffling strategies, the paper presents the first convergence and complexity guarantees for the Stochastic Reshuffling Mirror Descent method under such settings. Notably, DKC overcomes the limitations of classical smoothness assumptions while preserving closure under commonly used kernel functions and affine or conic combinations, thereby offering a verifiable and practical analytical tool for nonconvex relatively smooth optimization problems.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Kernel Methods

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

The global Lipschitz smoothness condition underlies most convergence and complexity analyses via two key consequences: the descent lemma and the gradient Lipschitz continuity. How to study the performance of optimization algorithms in the absence of Lipschitz smoothness remains an active area. The relative smoothness framework from Bauschke-Bolte-Teboulle (2017) and Lu-Freund-Nesterov (2018) provides an extended descent lemma, ensuring convergence of Bregman-based proximal gradient methods and their vanilla stochastic counterparts. However, many widely used techniques (e.g., momentum schemes, random reshuffling, and variance reduction) additionally require the Lipschitz-type bound for gradient deviations, leaving their analysis under relative smoothness an open area. To resolve this issue, we introduce the dual kernel conditioning (DKC) regularity condition to regulate the local relative curvature of the kernel functions. Combined with the relative smoothness, DKC provides a dual Lipschitz continuity for gradients: even though the gradient mapping is not Lipschitz in the primal space, it preserves Lipschitz continuity in the dual space induced by a mirror map. We verify that DKC is widely satisfied by popular kernels and is closed under affine composition and conic combination. With these novel tools, we establish the first complexity bounds as well as the iterate convergence of random reshuffling mirror descent for constrained nonconvex relative smooth problems.

Problem

• relative smoothness
• random reshuffling
• mirror descent
• nonconvex optimization
• Lipschitz continuity

Innovation

• Dual Kernel Conditioning
• Relative Smoothness
• Random Reshuffling
• Mirror Descent
• Nonconvex Optimization