Oracle-Efficient Online Classification with Stochastic Inputs and Adversarial Outputs

2026-09-27

AI Summary

This study addresses the fundamental challenge of balancing computational efficiency with regret minimization in online classification under stochastic inputs and adversarial outputs, resolving a long-standing open problem posed by Lazaric. We propose a Follow-the-Perturbed-Leader (FTPL) algorithm based on Gaussian perturbations that requires only a single oracle call per round. By reducing the computational complexity of mixed classification to its statistical learning counterpart and conducting rigorous analysis via VC dimension theory, we demonstrate that the proposed algorithm achieves an optimal expected regret bound of $\widetilde{O}(\sqrt{T \cdot \text{VC}(\mathcal{H})})$. This work resolves the aforementioned open problem and establishes a new paradigm for computationally efficient online learning.

Abstract

We consider contextual binary prediction with i.i.d. contexts from an unknown distribution and adaptively chosen losses. We show that a simple Follow-the-Perturbed-Leader algorithm with Gaussian perturbation for each observed context achieves the optimal $\widetilde O(\sqrt{T\log N})$ expected regret for a class of $N$ experts, while requiring one optimization-oracle call per round and no explicit enumeration of the class. For an infinite hypothesis class $\mathcal H$, the algorithm attains $\widetilde O(\sqrt{T\operatorname{VC}(\mathcal H)})$ regret. This resolves an open problem posed by Lazaric and Munos (2012), showing that hybrid classification is computationally as easy as statistical learning.

Problem

• online classification
• contextual binary prediction
• oracle-efficient learning
• regret minimization
• hybrid classification

Innovation

• Oracle-efficient online learning
• Follow-the-Perturbed-Leader
• Gaussian perturbation
• Contextual binary prediction
• Regret minimization