🤖 AI Summary
This paper addresses the loose dynamic regret bounds of Follow-the-Regularized-Leader (FTRL) in dynamic online convex optimization (OCO), identifying the root cause as the decoupling between state updates and iterates—not the projection mechanism, as conventionally assumed. To overcome this, we propose a novel analytical framework integrating optimistic prediction of future costs with linearized gradient pruning over historical gradients. Our approach employs recursive regularization to tightly couple states and iterates, enabling loop-free optimistic design and continuous interpolation between greediness and agility. The framework recovers classical dynamic regret upper bounds as special cases, yields finer-grained control over regret terms, and achieves the optimal $O(sqrt{T})$ dynamic regret over compact domains—without increasing gradient queries or memory overhead.
📝 Abstract
We revisit the Follow the Regularized Leader (FTRL) framework for Online Convex Optimization (OCO) over compact sets, focusing on achieving dynamic regret guarantees. Prior work has highlighted the framework's limitations in dynamic environments due to its tendency to produce"lazy"iterates. However, building on insights showing FTRL's ability to produce"agile"iterates, we show that it can indeed recover known dynamic regret bounds through optimistic composition of future costs and careful linearization of past costs, which can lead to pruning some of them. This new analysis of FTRL against dynamic comparators yields a principled way to interpolate between greedy and agile updates and offers several benefits, including refined control over regret terms, optimism without cyclic dependence, and the application of minimal recursive regularization akin to AdaFTRL. More broadly, we show that it is not the lazy projection style of FTRL that hinders (optimistic) dynamic regret, but the decoupling of the algorithm's state (linearized history) from its iterates, allowing the state to grow arbitrarily. Instead, pruning synchronizes these two when necessary.