🤖 AI Summary
This paper investigates the convexity conditions of optimization curves—i.e., the sequence of objective values {f(xₙ)}—generated by constant-step-size gradient descent on convex, L-smooth functions. The central question is: when is this sequence convex in the discrete sense, i.e., when are its second forward differences nonnegative—or equivalently, when is the forward difference f(xₙ) − f(xₙ₊₁) nonincreasing? The authors prove that strict convexity of the optimization curve holds if and only if the step size satisfies η ≤ 1.75/L, and this bound is tight (a counterexample shows convexity fails for any η > 1.75/L). Moreover, under the milder condition η ≤ 2/L, the gradient norm ∥∇f(xₙ)∥ is guaranteed to be nonincreasing. Leveraging forward-difference analysis, worst-case construction, and analogy with continuous-time gradient flow, the work establishes the first exact step-size characterizations for convexity and monotonicity in discrete optimization dynamics—thereby bridging a fundamental gap between discrete and continuous dynamical perspectives on convexity preservation.
📝 Abstract
We study when the emph{optimization curve} of first--order methods -- the sequence ${f(x_n)}*{nge0}$ produced by constant--stepsize iterations -- is convex, equivalently when the forward differences $f(x_n)-f(x*{n+1})$ are nonincreasing. For gradient descent (GD) on convex $L$--smooth functions, the curve is convex for all stepsizes $ηle 1.75/L$, and this threshold is tight. Moreover, gradient norms are nonincreasing for all $ηle 2/L$, and in continuous time (gradient flow) the curve is always convex. These results complement and refine the classical smooth convex optimization toolbox, connecting discrete and continuous dynamics as well as worst--case analyses.