Problem
Research questions and friction points this paper is trying to address.
Bernoulli multi-armed bandits
minimax solutions
worst-case regret
finite-horizon
single-arm bandit
Multi-Armed Bernoulli Bandits via Minimax Single-Arm Stopping
🤖 AI Summary
本文通过解决单臂伯努利赌博机问题，提出了一种基于索引的策略来处理有限时间范围内的多臂伯努利赌博机问题，并给出了无分布后悔界。
📝 Abstract
We develop an index policy for finite-horizon Bernoulli multi-armed bandits from minimax solutions to single-arm bandit (SAB) problems. Each SAB problem involves choosing between an unknown Bernoulli arm and a known reward. We show that minimizing worst-case regret of SAB problems over all non-anticipative policies admits an exact semi-infinite linear programming formulation. The resulting stopping policies offer a natural way to compare arms: the higher the known reward against which a policy continues sampling, the more promising the unknown arm. We turn this intuition into indices based on cumulative continuation probabilities, with a monotone adjustment and a reward-shortfall cap. By relating index errors to the regret of single-arm stopping policies, we establish a distribution-free regret bound of $4.45\sqrt{KT}+10.75K$ for $K$ arms and horizon $T$. This bound matches the minimax-optimal regret order established in the literature. The guarantee extends to rewards supported on $[0,1]$ through Bernoulli randomization. We also provide a finite-grid implementation with quantified approximation loss. In numerical experiments, the SAB-based index policy achieves lower worst-case regret than every tested benchmark policy across all evaluated numbers of arms and horizons, while closely matching the grid-based MAB minimax policy in the two-arm setting.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Minimax Solutions
Single-Arm Bandit Problems
Semi-Infinite Linear Programming
Cumulative Continuation Probabilities
🔎 Similar Papers
No similar papers found.
