CAAL: Contextual Bandits based Online Hand-Craft Active Learning Strategy Selection

2026-06-05

AI Summary

This work addresses the challenge in active learning that the uncertainty in the distribution of unlabeled data complicates the selection of an optimal hand-crafted querying strategy. To overcome this limitation, the authors propose a novel online strategy selection framework based on contextual multi-armed bandits, which— for the first time—incorporates external contextual information into active learning. The framework dynamically predicts the reward of each querying strategy during batch annotation and allows domain knowledge to be integrated through customizable context representations and reward functions. Evaluated across multiple public datasets, the proposed method significantly outperforms existing adaptive baselines and maintains consistent performance across varying batch sizes, thereby enhancing both the adaptability of strategy selection and the efficiency of annotation.

Abstract

The challenge with active learning algorithms is the uncertainty of the statistical distribution of unlabeled data, making it difficult to choose the best hand-crafted strategy. To address this, we introduced Contextual Adaptive Active Learning (CAAL). In CAAL, each "arm" represents a hand-crafted strategy. Unlike existing frameworks that select strategies based only on feedback from labeled data, we dynamically choose strategies for labeling batches of data using reward prediction with external context information. This general framework allows for customization with domain knowledge to design more effective rewards and context candidates. In addition, we experimentally show that CAAL outperforms the existing baseline adaptive strategy on public datasets using our reward and context design. Our results are consistent regardless of batch size in each iteration.

Problem

• active learning
• strategy selection
• statistical distribution
• unlabeled data
• hand-crafted strategy

Innovation

• Contextual Bandits
• Active Learning
• Strategy Selection
• Reward Prediction
• Online Learning