Asymptotically Optimal Linear Best Feasible Arm Identification with Fixed Budget

2025-06-03

AI Summary

This paper studies best feasible arm identification in linear multi-armed bandits under a fixed budget, aiming for exponential decay of the misidentification probability. Addressing the gap that prior work fails to characterize the exact exponential convergence rate of error probability under Gaussian noise, we establish, for the first time in the constrained linear setting, an algorithm achieving the information-theoretic lower bound on the optimal exponential decay rate. To this end, we propose a novel posterior sampling framework based on minimax game-theoretic sampling, integrating Thompson sampling principles, dynamically updated min/max learners, and rigorous information-theoretic analysis. We prove that the misidentification probability converges at the optimal exponential rate. Empirical evaluations demonstrate that our method significantly outperforms existing baselines across diverse complex instances, delivering simultaneous improvements in both accuracy and computational efficiency.

Technology Category

Machine Learning: Online Learning & BanditsSearch and Optimization: Sampling/Simulation-based SearchReasoning under Uncertainty: Stochastic Optimization

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingEconomics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

The challenge of identifying the best feasible arm within a fixed budget has attracted considerable interest in recent years. However, a notable gap remains in the literature: the exact exponential rate at which the error probability approaches zero has yet to be established, even in the relatively simple setting of $K$-armed bandits with Gaussian noise. In this paper, we address this gap by examining the problem within the context of linear bandits. We introduce a novel algorithm for best feasible arm identification that guarantees an exponential decay in the error probability. Remarkably, the decay rate -- characterized by the exponent -- matches the theoretical lower bound derived using information-theoretic principles. Our approach leverages a posterior sampling framework embedded within a game-based sampling rule involving a min-learner and a max-learner. This strategy shares its foundations with Thompson sampling, but is specifically tailored to optimize the identification process under fixed-budget constraints. Furthermore, we validate the effectiveness of our algorithm through comprehensive empirical evaluations across various problem instances with different levels of complexity. The results corroborate our theoretical findings and demonstrate that our method outperforms several benchmark algorithms in terms of both accuracy and efficiency.

Problem

• Establish exponential error decay rate for best feasible arm identification
• Develop optimal algorithm for linear bandits with fixed budget
• Validate algorithm performance against benchmark methods empirically

Innovation

• Novel algorithm for best feasible arm identification
• Posterior sampling with game-based min-max learners
• Exponential error decay matching theoretical lower bound