A General Framework for Clustering and Distribution Matching with Bandit Feedback

2024-09-08

arXiv.org

AI Summary

This paper studies online clustering with feedback: given $K$ unknown bandit arms drawn from $M$ latent clusters (arms in the same cluster share an identical underlying distribution), the goal is to exactly recover the clustering structure with minimal expected arm pulls and failure probability at most $delta$. This constitutes the first unified bandit framework modeling pairwise matching, anomaly detection, and multi-class clustering. We derive a non-asymptotic information-theoretic lower bound on sample complexity. We propose a novel Track-and-Stop algorithm integrated with Frank–Wolfe optimization, achieving asymptotically optimal sample complexity—the first such result. Moreover, we provide a fine-grained characterization of the convergence rate as $delta o 0$. Our algorithm attains theoretical optimality across classical problems including odd-arm identification, pairwise discovery, and $N$-ary clustering.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We develop a general framework for clustering and distribution matching problems with bandit feedback. We consider a $K$-armed bandit model where some subset of $K$ arms is partitioned into $M$ groups. Within each group, the random variable associated to each arm follows the same distribution on a finite alphabet. At each time step, the decision maker pulls an arm and observes its outcome from the random variable associated to that arm. Subsequent arm pulls depend on the history of arm pulls and their outcomes. The decision maker has no knowledge of the distributions of the arms or the underlying partitions. The task is to devise an online algorithm to learn the underlying partition of arms with the least number of arm pulls on average and with an error probability not exceeding a pre-determined value~$delta$. Several existing problems fall under our general framework, including finding $M$ pairs of arms, odd arm identification, and $N$-ary clustering of $K$ arms belong to our general framework. We derive a non-asymptotic lower bound on the average number of arm pulls for any online algorithm with an error probability not exceeding $delta$. Furthermore, we develop a computationally-efficient online algorithm based on the Track-and-Stop method and Frank--Wolfe algorithm, and show that the average number of arm pulls of our algorithm asymptotically matches that of the lower bound. Our refined analysis also uncovers a novel bound on the speed at which the average number of arm pulls of our algorithm converges to the fundamental limit as $delta$ vanishes.

Problem

• Optimal Strategy
• Anomaly Detection
• Multi-class Classification

Innovation

• Multi-armed Bandit Problem
• Minimization of Trials
• Theoretical Boundaries