🤖 AI Summary
This paper addresses the decision-theoretic analysis of multi-armed bandits under local asymptotic conditions. Methodologically, it introduces a unified Bayesian and minimax risk modeling framework: for the first time, the minimal Bayesian risk is characterized as the solution to a second-order partial differential equation (PDE), rigorously derived via diffusion approximation and limit experiment theory; asymptotically sufficient state variables are identified, enabling intrinsic dimension reduction and yielding explicit optimal Bayesian and minimax strategies. Theoretically, the PDE solution is proven asymptotically universal—valid under both parametric and nonparametric reward distributions. Empirically, the proposed strategies achieve substantially lower risk than Thompson sampling, with reductions of up to 50%, thereby significantly improving experimental efficiency and decision reliability.
📝 Abstract
We provide a decision theoretic analysis of bandit experiments under local asymptotics. Working within the framework of diffusion processes, we define suitable notions of asymptotic Bayes and minimax risk for these experiments. For normally distributed rewards, the minimal Bayes risk can be characterized as the solution to a second-order partial differential equation (PDE). Using a limit of experiments approach, we show that this PDE characterization also holds asymptotically under both parametric and non-parametric distributions of the rewards. The approach further describes the state variables it is asymptotically sufficient to restrict attention to, and thereby suggests a practical strategy for dimension reduction. The PDEs characterizing minimal Bayes risk can be solved efficiently using sparse matrix routines or Monte-Carlo methods. We derive the optimal Bayes and minimax policies from their numerical solutions. These optimal policies substantially dominate existing methods such as Thompson sampling; the risk of the latter is often twice as high.
