Sampling Allocation of LinUCB: Optimal Design Limits in the Small-Gap Regime

2026-10-05

AI Summary

This study addresses the previously unclear intrinsic connection between the sample allocation mechanism of the LinUCB algorithm and optimal experimental design in the small-gap regime. By introducing a mean-field perspective that integrates asymptotic statistical inference with regularized least squares estimation, this work systematically characterizes the limiting behavior of the empirical sampling distribution of LinUCB and rigorously proves its convergence to the D-optimal design set. Consequently, it achieves a theoretical unification of regret minimization in online learning and information-efficient experimental design. Furthermore, this paper precisely identifies the asymptotic constant of regret and establishes a central limit theorem for reward parameter estimation, thereby providing a novel paradigm for the asymptotic analysis of linear bandits.

Abstract

We study the sampling allocation of LinUCB in the small-gap regime, where the reward gaps are of order at most $n^{-1/2}$ over the decision horizon $n$. This scaling captures the hard instances underlying worst-case regret lower bounds, for which LinUCB is known to be near optimal up to logarithmic factors in $n$. Using a mean-field perspective, we characterize this allocation through the empirical sampling distribution, a macroscopic object that averages the effect of adaptive decisions over the horizon, and identify its limit as $n\to\infty$. We establish that in this regime, the empirical sampling distribution induced by LinUCB converges to the set of D-optimal designs. This central result reveals that, in the small-gap regime, LinUCB not only achieves near optimal minimax regret but also allocates samples in a way that is asymptotically efficient for learning the reward parameter, thereby connecting regret-driven online learning with information-efficient experimental design. Building on the optimal design limit, we obtain two useful consequences. First, we refine the asymptotic regret analysis of LinUCB in the small-gap regime by characterizing its leading-order constant in the limit. Second, we show that, despite LinUCB's adaptive sampling strategy, the regularized least-squares estimator satisfies a central-limit-type theorem in the small-gap regime, thereby enabling valid statistical inference for the reward parameter.

Problem

• LinUCB
• sampling allocation
• small-gap regime
• D-optimal design
• empirical sampling distribution

Innovation

• LinUCB
• Small-Gap Regime
• D-Optimal Design
• Sampling Allocation
• Central Limit Theorem