Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability

2026-10-01

AI Summary

This study addresses the unclear relationship between transition time bounds, dimensionality, and step size in edge-of-stability dynamics for separable logistic regression. By employing mathematical induction to control variations in gradient-dominant samples, combined with constructive proofs and optimization-theoretic analysis, this work constructs matching hard instances to derive worst-case bounds in arbitrary dimensions. The results refute the prevailing conjecture that transition time is independent of step size, revealing instead a logarithmic dependence and a coupling effect between dimensionality and sample size. Furthermore, this research rigorously establishes that the tight bound on worst-case transition time is Θ((log η)^(min{n-2,d-2})), thereby providing a precise theoretical characterization of these dynamical mechanisms.

Abstract

We study logistic regression on linearly separable data under gradient descent with a large constant stepsize $η$. Such dynamics may exhibit a characteristic Edge of Stability phenomenon, in which the loss initially oscillates before transitioning to a stable phase of monotone decrease. Existing work provides a tight $Θ(1)$ bound in dimension $d=2$ as $η\to \infty$ and conjectures a bound independent of $η$ in arbitrary dimensions $d\geq 2$. In this paper, we disprove this conjecture by showing that, for every fixed sample size $n\geq 2$ and sufficiently small margin $γ$, the worst-case transition time is $$Θ\!\left((\logη)^{\min\{n-2,d-2\}}\right)$$ uniformly over $d\geq2$. The key challenge in establishing a tight bound is that the sample contributing most strongly to the gradient can change repeatedly across iterations. To address this issue, we control such changes by induction on dimension and sample size, and construct matching hard instances.

Problem

• Logistic Regression
• Edge of Stability
• Transition Time Bounds
• Gradient Descent
• Linearly Separable Data

Innovation

• Edge of Stability
• Logistic Regression
• Transition Time Bounds
• Gradient Descent
• Linearly Separable Data