🤖 AI Summary
This work addresses worst-case online optimization, unifying the study of online convex and non-convex optimization over Euclidean and non-Euclidean domains—including simplices and matrix manifolds—under a regret minimization framework. We propose a parameter-free, adaptive algorithmic framework that supports unbounded decision sets and unknown gradient magnitudes. Unifying online mirror descent (OMD) and follow-the-regularized-leader (FTRL), we reformulate first- and second-order methods and, for the first time, integrate convex surrogate losses, randomization schemes, and multi-armed bandit feedback—both adversarial and stochastic—into this coherent paradigm. Our theoretical analysis is self-contained, elementary, and accessible without prerequisites; all algorithms achieve tight, optimal regret bounds. The resulting framework establishes a universal, concise, and pedagogically transparent foundation for modern online learning, substantially lowering both theoretical barriers and practical implementation complexity.
📝 Abstract
In this monograph, I introduce the basic concepts of Online Learning through a modern view of Online Convex Optimization. Here, online learning refers to the framework of regret minimization under worst-case assumptions. I present first-order and second-order algorithms for online learning with convex losses, in Euclidean and non-Euclidean settings. All the algorithms are clearly presented as instantiation of Online Mirror Descent or Follow-The-Regularized-Leader and their variants. Particular attention is given to the issue of tuning the parameters of the algorithms and learning in unbounded domains, through adaptive and parameter-free online learning algorithms. Non-convex losses are dealt through convex surrogate losses and through randomization. The bandit setting is also briefly discussed, touching on the problem of adversarial and stochastic multi-armed bandits. These notes do not require prior knowledge of convex analysis and all the required mathematical tools are rigorously explained. Moreover, all the proofs have been carefully chosen to be as simple and as short as possible.
