regret analysis

This paper studies adversarial multi-armed bandit optimization for nonconvex, nonsmooth functions, where the loss in each round comprises a linear term plus an arbitrary small perturbation adaptively chosen after observing the player’s action. We establish the first unified theoretical framework for adversarial bandits under approximately linear structure, integrating online gradient estimation, random directional sampling, adaptive perturbation analysis, and information-theoretic lower bound construction. Our analysis yields tight regret bounds: $O(sqrt{T})$ expected regret and $O(sqrt{T log T})$ high-probability regret, both matched by a $Omega(sqrt{T})$ lower bound. These results significantly improve upon prior high-probability analyses for linear bandits and provide novel upper bounds and theoretical guarantees for nonsmooth adversarial optimization.

This work addresses adversarial bandit optimization under a global perturbation budget, where in each round the loss consists of a linear function plus an action-dependent perturbation term, with the total perturbation constrained globally. Within this non-convex and non-smooth setting, the paper establishes—for the first time—both expected and high-probability regret upper bounds under a global perturbation budget, improving upon the classical high-probability regret bound in the unperturbed case. Additionally, it provides a matching lower bound on the expected regret. The analysis combines techniques from adversarial bandits, perturbation modeling, and refined probabilistic arguments, offering rigorous theoretical guarantees for online decision-making in perturbed environments.

This paper addresses the limitation of the Gradient Bandit algorithm—its reliance on the softmax assumption of action independence—which hinders modeling inter-action correlations. We propose a Generalized Gradient Bandit framework that integrates discrete choice theory, particularly the Generalized Nested Logit model, with online learning. Our method introduces a generalized gradient update rule and a closed-form sampling mechanism that explicitly capture action dependencies, supporting both stochastic and adversarial environments. The key contribution is the first incorporation of nested structure into gradient bandit algorithms, thereby relaxing the classical i.i.d. and independence assumptions inherent in softmax-based policies and enabling efficient cooperative learning among correlated actions. We establish a sublinear regret bound theoretically. Empirical results on stochastic multi-armed bandit tasks demonstrate significantly improved convergence speed and decision quality, while preserving modeling flexibility and computational efficiency.

This paper addresses the problem of safely and efficiently improving online learning performance for stochastic multi-armed bandits (MAB) and combinatorial MAB under offline–online distribution shift. To tackle distribution mismatch between offline and online data, we propose MIN-UCB—a novel adaptive algorithm that automatically discards low-quality offline data when the shift magnitude is unknown, ensuring safety, and actively leverages bounded-shift offline information to substantially reduce regret when the shift is bounded. We establish, for the first time, a theoretical lower bound on regret for MAB with biased offline data. We prove that MIN-UCB achieves tight instance-dependent and instance-independent regret bounds—strictly outperforming classical UCB. Both theoretical analysis and numerical experiments demonstrate its robustness and superiority across diverse shift regimes.

This work addresses worst-case online optimization, unifying the study of online convex and non-convex optimization over Euclidean and non-Euclidean domains—including simplices and matrix manifolds—under a regret minimization framework. We propose a parameter-free, adaptive algorithmic framework that supports unbounded decision sets and unknown gradient magnitudes. Unifying online mirror descent (OMD) and follow-the-regularized-leader (FTRL), we reformulate first- and second-order methods and, for the first time, integrate convex surrogate losses, randomization schemes, and multi-armed bandit feedback—both adversarial and stochastic—into this coherent paradigm. Our theoretical analysis is self-contained, elementary, and accessible without prerequisites; all algorithms achieve tight, optimal regret bounds. The resulting framework establishes a universal, concise, and pedagogically transparent foundation for modern online learning, substantially lowering both theoretical barriers and practical implementation complexity.

This work addresses the challenge of integrating offline data with online learning in stochastic linear bandits by proposing a novel algorithm that initially leverages offline data to form a prior and subsequently enhances online exploration in an adaptive manner, dynamically balancing the contributions of both sources. Within the structured linear bandit setting, the method is the first to simultaneously outperform purely online and purely offline strategies, achieving a sublinear regret bound with respect to the optimal action. Notably, the regret decreases as the amount of offline data increases. Theoretical analysis establishes a rigorous upper bound on regret, and extensive experiments demonstrate that the proposed approach significantly outperforms existing baselines across various settings.

We study the distribution of regret in stochastic multi-armed bandits and episodic reinforcement learning through a unified framework. We formalize a distributional regret bound as a probabilistic guarantee that holds uniformly over all confidence levels $δ\in (0,1]$, thereby characterizing the regret distribution across the full range of $δ$. We present a simple UCBVI-style algorithm with exploration bonus $\min\{c_{1,k}/N, c_{2,k}/\sqrt{N}\}$, where $N$ denotes the visit count and $(c_{1,k},c_{2,k})$ are user-specified parameters. For arbitrary parameter sequences, we derive general gap-independent and gap-dependent distributional regret bounds, yielding a principled characterization of how the parameters control the trade-off between expected performance, tail risk, and instance-dependent behavior. In particular, our bounds achieve optimal trade-offs between expected and distributional regret in both minimax and instance-dependent regimes. As a special case, for multi-armed bandits with $A$ arms and horizon $T$, we obtain a distributional regret bound of order $\mathcal{O}(\sqrt{AT}\log(1/δ))$, confirming the conjecture of Lattimore & Szepesvári (2020, Section 17.1) for the first time.

This work addresses bandit optimization with β-smooth convex losses under adversarial perturbations subject to a global budget constraint, where perturbations are adaptively generated after the learner selects an action and their cumulative magnitude is bounded by a total budget. The authors generalize existing perturbation models—previously limited to linear losses—to general β-smooth convex losses, proposing an improved standard bandit algorithm. Through a refined analysis of the additional regret introduced by the perturbations, they establish the first explicit expected regret upper bound that depends directly on the perturbation budget. Notably, this bound naturally recovers the known regret guarantees for classical β-smooth bandit convex optimization in the absence of perturbations, thereby unifying the perturbed and unperturbed settings within a single theoretical framework.

This work investigates the fundamental learning speed limits of mean-based algorithms under bandit feedback with an unknown time horizon, and explores their relationship with no-regret algorithms. We propose two novel algorithms: one generalizing the ε-greedy strategy and another extending the mean-based Exp3 algorithm to settings with unknown horizon. Our theoretical contributions include establishing, for the first time, a lower bound on the γₜ sequence in mean-based algorithms, thereby proving the existence of an inherent limit on learning speed, and revealing a non-trivial intersection between mean-based and no-regret algorithms. Empirical results demonstrate that although our proposed algorithms converge slightly slower, they achieve competitive performance, confirming the feasibility of simultaneously satisfying both mean-based estimation and no-regret properties.