Last-Iterate Convergence of Optimistic Multiplicative Weight Update

2026-06-10

AI Summary

This work resolves a long-standing open problem regarding the last-iterate convergence of the Optimistic Multiplicative Weights Update (OMWU) algorithm in smooth convex-concave saddle-point problems. By integrating non-Euclidean optimization frameworks, convex analysis, and dynamical systems theory, we establish for the first time that, under a sufficiently small fixed learning rate, OMWU converges in its last iterate to a saddle point without requiring common assumptions such as solution uniqueness, strict complementarity, error bounds, or initialization near the solution set. The key innovation lies in a novel boundary argument that verifies all limit points satisfy the KKT inequalities for inactive coordinates, thereby proving last-iterate convergence of OMWU in general smooth convex-concave settings.

Abstract

Optimistic Gradient Descent Ascent (OGDA) and Optimistic Multiplicative-Weights Update (OMWU) are two very popular algorithms to solve convex/concave saddle-point problems, where OMWU is the non-Euclidean, entropic version of OGDA. It is known since the '80s that the last iterate of OGDA asymptotically converges to a saddle point in smooth problems. On the other hand, it is unknown if OMWU has the same property. In this paper, I show that OMWU converges asymptotically for smooth convex-concave saddle-point problems, with a small enough constant learning rate. The result does not require uniqueness, strict complementarity, an error bound, or initialization near a solution. The main new ingredient is a boundary argument showing that every cluster point satisfies the inactive-coordinate KKT inequalities. The boundary argument was discovered with assistance from ChatGPT and is documented in the appendix.

Problem

• Optimistic Multiplicative-Weights Update
• last-iterate convergence
• convex-concave saddle-point problems
• asymptotic convergence

Innovation

• Optimistic Multiplicative-Weights Update
• last-iterate convergence
• saddle-point problems
• boundary argument
• KKT inequalities