A Unified Optimism-Agnostic Framework for Linear Bandits over Spherical Action Sets

2026-09-26

AI Summary

This study addresses the disconnect between parameter estimation quality and regret bounds in linear bandits, as well as the limitations of traditional elliptic potential function analysis. To this end, it proposes an algorithm-agnostic, modular analytical framework. By employing high-probability analysis techniques for non-elliptic potential functions, combined with eigenvalue theory for design matrices and the geometric properties of action sets, this work explicitly establishes, for the first time, the intrinsic connection between minimum eigenvalue growth conditions and the optimal $\mathcal{O}(\sqrt{T}\log T)$ regret rate. The analysis confirms that both UCB and Thompson Sampling variants achieve optimal regret bounds under spherical action sets. Ultimately, this research provides a unified theoretical analysis tool for linear bandit algorithms.

Abstract

Linear bandits model sequential decision-making problems with noisy rewards that are linear in the decision variable, where an agent must simultaneously learn about an unknown parameter that governs the mean rewards, while maximizing (expected) rewards over time. Two prominent algorithmic families--upper confidence bound (UCB) and Thompson sampling (TS)--achieve a balance of exploration (to estimate said parameter) and exploitation (utilization of knowledge about it) across time. The quality of estimation of that parameter depends on the eigenvalues of a design matrix. In this paper, we begin by showing that if the inference quality obtained from exploration, encoded in the minimum eigenvalue of the design matrix, grows $\gtrsim \sqrt{t}$ with time $t$, while actions remain sufficiently concentrated for exploitation, then an algorithm produces optimal high-probability $\mathcal{O}(\sqrt{T}\log T)$-regret rate over a time-horizon $T$ for spherical action sets. This analysis is algorithm-agnostic and follows an alternative route to the classical optimism-based elliptical-potential argument for regret analysis. Then, we illustrate that variants of UCB and TS satisfy the inference and concentration properties and in turn, enjoy optimal regret rate. In effect, our results provide a modular framework that can be used to analyze linear bandit algorithms and explicitly connect quality of parameter estimation to optimal regret accumulation.

Problem

• Linear Bandits
• Regret Analysis
• Spherical Action Sets
• Optimism-Agnostic Framework
• Exploration-Exploitation

Innovation

• Linear Bandits
• Algorithm-Agnostic Framework
• Regret Analysis
• Spherical Action Sets
• Design Matrix Eigenvalues