Representative Action Selection for Large Action-Space Meta-Bandits

2025-05-23

AI Summary

For meta-multi-armed bandits with large action spaces, this paper proposes an action representative set selection framework that approximates full-action-space performance while drastically reducing the number of actions. Methodologically, it introduces the first integration of Gaussian process–based action similarity modeling with ε-net sampling to construct a theoretically grounded and computationally efficient selection mechanism, further enhanced by meta-learning for constrained bandit policy optimization. We establish a tight theoretical upper bound on the performance loss induced by the selected subset. Empirically, the approach retains over 95% of the original reward under 90% action compression and consistently outperforms Thompson Sampling and UCB across multiple benchmarks. The core contribution is the first action space compression paradigm for bandits that simultaneously offers rigorous theoretical guarantees and practical efficacy.

Technology Category

• Machine Learning: Online Learning & Bandits
• Multiagent Systems: Mechanism Design
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We study the problem of selecting a subset from a large action space shared by a family of bandits, with the goal of achieving performance nearly matching that of using the full action space. We assume that similar actions tend to have related payoffs, modeled by a Gaussian process. To exploit this structure, we propose a simple epsilon-net algorithm to select a representative subset. We provide theoretical guarantees for its performance and compare it empirically to Thompson Sampling and Upper Confidence Bound.

Problem

• Selecting subset from large action space for bandits
• Achieving performance close to full action space
• Modeling similar actions with Gaussian process

Innovation

• Uses Gaussian process for payoff modeling
• Proposes epsilon-net algorithm for selection
• Compares with Thompson Sampling and UCB