A Unified Lyapunov-IQC Framework for Uniform Stability of Smooth Quadratic First-Order Accelerated Optimizers

2026-05-08

AI Summary

This work addresses the challenge of analyzing uniform stability for first-order accelerated optimization algorithms with momentum—such as Nesterov’s accelerated gradient method—under β-smoothness and γ-strong convexity assumptions. It introduces, for the first time, the Lyapunov–Integral Quadratic Constraint (Lyapunov-IQC) framework from robust control into optimization theory. By modeling the accelerated dynamics as a Lur’e-type feedback system and combining Lyapunov functions with IQCs, the stability verification problem is transformed into a computable linear matrix inequality (LMI), solvable via semidefinite programming. This approach not only recovers classical uniform stability bounds but also offers a modular and extensible pathway for certifying stability, thereby establishing a structural bridge between optimization algorithms and control theory.

Abstract

We develop a unified Lyapunov-integral quadratic constraint (IQC) framework for establishing uniform stability of first-order accelerated optimization algorithms in the $β$-smooth and $γ$-strongly convex regime. Classical analyses of uniform stability, such as the work of Hardt, Recht, and Singer for stochastic gradient descent (SGD), rely on direct coupling arguments and case-by-case control of iterate differences under random sampling. Extending such arguments to accelerated methods, such as Nesterov Accelerated Gradient (NAG), is complicated by the presence of higher-order state dynamics induced by momentum. We first extend this classical approach with the use of Lyapunov functions to provide a uniform stability bound for smooth quadratic NAG, and supplement this result with small-scale numerical experiments. We then extend this framework by modeling first-order accelerated optimizers as Lur'e-type feedback interconnections between a linear dynamical system and a (non-linear) gradient operator. $β$-Smoothness and $γ$-strong convexity are encoded a sector IQC inequality. Under this representation, uniform stability is certified via the existence of a quadratic Lyapunov function satisfying a finite-dimensional linear matrix inequality (LMI) in the form of a feasibility problem, which can be solved via semi-definite programming (SDP). We instantiate this framework for NAG and show how classical uniform stability bounds can be recovered via this framework. These results underscore a structural connection between optimization dynamics and robust control theory, providing a modular methodology for reliable and reproducible numerical certification of uniform stability and generalization behavior of first-order methods via convex optimization tools that is adaptable to increasingly complex optimization algorithms.

Problem

• uniform stability
• accelerated optimization
• first-order methods
• smoothness
• strong convexity

Innovation

• Lyapunov-IQC framework
• uniform stability
• accelerated optimization
• linear matrix inequality (LMI)
• robust control