🤖 AI Summary
This work identifies function continuity—not an inherent trade-off—as the fundamental cause of the robustness-accuracy dilemma in machine learning. Theoretically, it provides the first explanation of adversarial examples through the lens of function continuity and uniform convergence. Methodologically, it introduces a novel paradigm of piecewise training with heterogeneous continuity assumptions, and constructs a learnable framework grounded in harmonic analysis and complex function theory. Experiments demonstrate that models adopting discontinuous hypotheses consistently outperform those assuming continuity, achieving superior robust accuracy (under adversarial attacks) and natural accuracy (on clean data). This work thus establishes a new theoretical foundation and practical pathway for designing classifiers that simultaneously attain high robustness and high accuracy.
📝 Abstract
The reliability of a learning model is key to the successful deployment of machine learning in various applications. Creating a robust model, particularly one unaffected by adversarial attacks, requires a comprehensive understanding of the adversarial examples phenomenon. However, it is difficult to describe the phenomenon due to the complicated nature of the problems in machine learning. It has been shown that adversarial training can improve the robustness of the hypothesis. However, this improvement comes at the cost of decreased performance on natural samples. Hence, it has been suggested that robustness and accuracy of a hypothesis are at odds with each other. In this paper, we put forth the alternative proposal that it is the continuity of a hypothesis that is incompatible with its robustness and accuracy. In other words, a continuous function cannot effectively learn the optimal robust hypothesis. To this end, we will introduce a framework for a rigorous study of harmonic and holomorphic hypothesis in learning theory terms and provide empirical evidence that continuous hypotheses does not perform as well as discontinuous hypotheses in some common machine learning tasks. From a practical point of view, our results suggests that a robust and accurate learning rule would train different continuous hypotheses for different regions of the domain. From a theoretical perspective, our analysis explains the adversarial examples phenomenon as a conflict between the continuity of a sequence of functions and its uniform convergence to a discontinuous function.