Formal Models of Active Learning from Contrastive Examples

2025-06-18

AI Summary

This work investigates how contrastive examples—samples with differing labels but similar semantics—affect sample complexity in active learning. Method: We develop the first formal contrastive-driven active learning theoretical framework for geometric concept classes and Boolean function classes, integrating computational learning theory, sample complexity analysis, and concept class modeling to rigorously characterize the relationship between contrastive example selection and learning efficiency. Contribution/Results: We establish an intrinsic connection between our model and the classical self-directed learning framework. Theoretical analysis demonstrates that judicious incorporation of contrastive examples can significantly reduce sample complexity, with the efficacy of distinct selection strategies precisely quantifiable. Our results provide provable theoretical guarantees and principled design guidelines for efficient human annotation, advancing active learning from empirical practice toward rigorous theoretical foundations.

Technology Category

• Machine Learning: Active Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Machine learning can greatly benefit from providing learning algorithms with pairs of contrastive training examples -- typically pairs of instances that differ only slightly, yet have different class labels. Intuitively, the difference in the instances helps explain the difference in the class labels. This paper proposes a theoretical framework in which the effect of various types of contrastive examples on active learners is studied formally. The focus is on the sample complexity of learning concept classes and how it is influenced by the choice of contrastive examples. We illustrate our results with geometric concept classes and classes of Boolean functions. Interestingly, we reveal a connection between learning from contrastive examples and the classical model of self-directed learning.

Problem

• Studies effect of contrastive examples on active learners
• Analyzes sample complexity in learning concept classes
• Explores connection between contrastive and self-directed learning

Innovation

• Theoretical framework for contrastive examples' impact
• Study sample complexity in concept classes
• Link contrastive learning to self-directed learning