π€ AI Summary
This paper investigates efficient PAC learning of halfspaces under constant-level malicious noiseβwhere both features and labels may be adversarially corrupted. Addressing the limitation of prior approaches, whose noise tolerance depends on either the target error Ξ΅ or the margin parameter, we propose the first polynomial-time algorithm achieving robust learning under a fixed constant noise rate. Our method introduces a carefully designed reweighting scheme with controllable gradient degradation, and establishes a novel robustness analysis framework based on a weighted hinge loss. Under joint assumptions of distributional regularity and a large-margin condition, we rigorously prove that this framework attains Ξ΅-accurate learning. Both theoretical analysis and empirical evaluation demonstrate that our approach breaks the longstanding coupling between noise tolerance and accuracy/margin requirements, achieving, for the first time, efficient and robust learnability under O(1) malicious noise rate.
π Abstract
Understanding noise tolerance of machine learning algorithms is a central quest in learning theory. In this work, we study the problem of computationally efficient PAC learning of halfspaces in the presence of malicious noise, where an adversary can corrupt both instances and labels of training samples. The best-known noise tolerance either depends on a target error rate under distributional assumptions or on a margin parameter under large-margin conditions. In this work, we show that when both types of conditions are satisfied, it is possible to achieve constant noise tolerance by minimizing a reweighted hinge loss. Our key ingredients include: 1) an efficient algorithm that finds weights to control the gradient deterioration from corrupted samples, and 2) a new analysis on the robustness of the hinge loss equipped with such weights.