Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity

2026-06-09

AI Summary

This work investigates the sensitivity of mirror descent to initial perturbations in non-Euclidean geometries and its implications for algorithmic robustness. In the setting of smooth, strongly convex optimization with non-quadratic regularization, the paper presents the first construction demonstrating that initial perturbations can be exponentially amplified to εe^{Ω(ηT)} (or polylog⁻¹(1/ε)) within T iterations, exposing instability of KL regularization—particularly in high-dimensional or boundary regions. To address this, the authors propose an anchored Bregman regularization mechanism that substantially enhances stability while preserving optimization performance. By integrating tools from convex optimization, Bregman divergences, and perturbation analysis of dynamical systems, the study systematically elucidates why mirror descent can be inherently more sensitive to initialization than gradient descent.

Abstract

Mirror Descent (MD) extends Gradient Descent (GD) beyond Euclidean geometry and has recently reappeared as a lens for KL-regularized policy optimization in reinforcement learning and LLM post-training. This raises a basic robustness question, crucial to reproducibility and reliability: how sensitive are MD dynamics to their inputs? We focus on initialization, often itself a pretrained or previously aligned model. Quadratic-regularized MD, including GD and Mahalanobis geometries, is well-known to be stable for convex smooth objectives. We show a sharp contrast: once the regularizer is non-quadratic, MD can be exponentially more sensitive to initialization than GD, even with a well-conditioned regularizer in Euclidean norm. We give a three-dimensional construction with a convex, smooth objective and a strongly convex, smooth, well-conditioned regularizer where an initial $\varepsilon$ perturbation is quickly amplified to $\min\{\text{polylog}^{-1}(1/\varepsilon), \varepsilon e^{Ω(ηT)}\}$ after $T$ iterations of MD with step size $η$. For canonical KL-regularized MD on the simplex, we show that even linear objectives can amplify an initial $\varepsilon$ perturbation exponentially fast in high-dimensional or near-boundary regimes. Finally, we show that adding a Bregman regularization term toward an anchor point can stabilize the dynamics while largely preserving the optimization guarantees, and that the choice of anchor is crucial: anchoring at the initialization only partially mitigates the instability, whereas anchoring at a fixed point yields a more stable mechanism.

Problem

• Mirror Descent
• initialization sensitivity
• robustness
• non-quadratic regularization
• optimization stability

Innovation

• Mirror Descent
• initialization sensitivity
• non-quadratic regularization
• Bregman regularization
• exponential instability