Constrained Best Arm Identification with Tests for Feasibility

2025-11-12

AI Summary

In best-arm identification (BAI) under feasibility constraints—where the goal is to identify the optimal arm among $K$ arms that satisfies $N$ safety constraints—existing methods assume simultaneous observation of reward and constraint outcomes. However, in applications such as drug discovery, performance and safety must be evaluated separately. Method: We propose the first “selectable-test-type” BAI framework, enabling adaptive selection at each round of whether to query the reward or any individual constraint of the chosen arm. Building on this, we design an adaptive sampling strategy that asymptotically minimizes sample complexity under a fixed confidence level $1-delta$. Contribution/Results: We prove that our algorithm achieves the information-theoretic lower bound on sample complexity. Experiments on synthetic and real-world datasets demonstrate that our method significantly outperforms state-of-the-art approaches in both accuracy and efficiency, while preserving rigorous statistical guarantees.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Best arm identification (BAI) aims to identify the highest-performance arm among a set of $K$ arms by collecting stochastic samples from each arm. In real-world problems, the best arm needs to satisfy additional feasibility constraints. While there is limited prior work on BAI with feasibility constraints, they typically assume the performance and constraints are observed simultaneously on each pull of an arm. However, this assumption does not reflect most practical use cases, e.g., in drug discovery, we wish to find the most potent drug whose toxicity and solubility are below certain safety thresholds. These safety experiments can be conducted separately from the potency measurement. Thus, this requires designing BAI algorithms that not only decide which arm to pull but also decide whether to test for the arm's performance or feasibility. In this work, we study feasible BAI which allows a decision-maker to choose a tuple $(i,ell)$, where $iin [K]$ denotes an arm and $ell$ denotes whether she wishes to test for its performance ($ell=0$) or any of its $N$ feasibility constraints ($ellin[N]$). We focus on the fixed confidence setting, which is to identify the extit{feasible} arm with the extit{highest performance}, with a probability of at least $1-delta$. We propose an efficient algorithm and upper-bound its sample complexity, showing our algorithm can naturally adapt to the problem's difficulty and eliminate arms by worse performance or infeasibility, whichever is easier. We complement this upper bound with a lower bound showing that our algorithm is extit{asymptotically ($delta ightarrow 0$) optimal}. Finally, we empirically show that our algorithm outperforms other state-of-the-art BAI algorithms in both synthetic and real-world datasets.

Problem

• Identifying the highest-performing arm under multiple feasibility constraints
• Deciding whether to test performance or feasibility for each arm
• Developing asymptotically optimal algorithms for constrained best arm identification

Innovation

• Tests performance and feasibility separately
• Adaptively eliminates arms by performance or infeasibility
• Asymptotically optimal sample complexity guarantees