Low-Budget Active Learning through Entropic Optimal Transport

2026-10-01

AI Summary

This study addresses the challenge of high-accuracy coreset selection under expensive annotation costs in low-budget active learning. To this end, it proposes a coreset selection framework based on entropy-regularized optimal transport within self-supervised feature spaces. By incorporating the Sinkhorn divergence, the method achieves dimension-independent sample complexity guarantees and enables efficient gradient computation. Furthermore, it combines gradient-based optimization with local search mechanisms to enhance selection quality. The proposed approach significantly outperforms existing heuristic strategies across both image and medical datasets, effectively improving model performance in low-budget scenarios.

Abstract

We consider low-budget active learning, which consists of selecting a limited number of points, the coreset, such that a model can be trained to high accuracy on the selection only. This problem is particularly relevant in contexts where labeling requires costly expert intervention, as in medical applications. We leverage features extracted from a pretrained self-supervised model to represent the data, and perform coreset selection directly in this feature space. In this paper, we use entropic optimal transport, specifically the Sinkhorn divergence, as the coreset selection criterion, which first allows us to get dimension-free sample complexity results, and second admits computationally efficient gradient evaluations. This opens the way to using gradient-based algorithms to rapidly compute solution candidates, further improved by a swap-based local search, with guarantees on the solution quality. Experiments on image benchmarks and medical datasets show that our method outperforms state-of-the-art heuristics in low-budget settings.

Problem

• Low-budget active learning
• Coreset selection
• Labeling cost
• Self-supervised features

Innovation

• Low-Budget Active Learning
• Entropic Optimal Transport
• Sinkhorn Divergence
• Coreset Selection
• Self-Supervised Features