🤖 AI Summary
To address the vulnerability of deep neural networks to adversarial attacks, this paper proposes BORT, a multi-branch robust training framework that enhances adversarial robustness from the perspective of deep feature distribution. Its core innovation is a branch orthogonality loss that enforces orthogonality among feature subspaces of parallel branches, thereby improving invariance to input perturbations. Crucially, BORT requires only the original training data—introducing no auxiliary data, inference overhead, or pre-trained modules. Integrated with ℓ∞-constrained adversarial training, BORT is comprehensively evaluated on CIFAR-10, CIFAR-100, and SVHN. On CIFAR-10 and CIFAR-100, it achieves robust accuracies of 67.3% and 41.5%, respectively—surpassing state-of-the-art methods by 7.23% and 9.07%. These results significantly outperform all existing data-free approaches and even surpass several methods relying on large-scale auxiliary datasets.
📝 Abstract
Deep neural networks (DNNs) are vulnerable to adversarial examples, in which DNNs are misled to false outputs due to inputs containing imperceptible perturbations. Adversarial training, a reliable and effective method of defense, may significantly reduce the vulnerability of neural networks and becomes the de facto standard for robust learning. While many recent works practice the data-centric philosophy, such as how to generate better adversarial examples or use generative models to produce additional training data, we look back to the models themselves and revisit the adversarial robustness from the perspective of deep feature distribution as an insightful complementarity. In this paper, we propose Branch Orthogonality adveRsarial Training (BORT) to obtain state-of-the-art performance with solely the original dataset for adversarial training. To practice our design idea of integrating multiple orthogonal solution spaces, we leverage a simple and straightforward multi-branch neural network that eclipses adversarial attacks with no increase in inference time. We heuristically propose a corresponding loss function, branch-orthogonal loss, to make each solution space of the multi-branch model orthogonal. We evaluate our approach on CIFAR-10, CIFAR-100, and SVHN against ell_{infty} norm-bounded perturbations of size epsilon = 8/255, respectively. Exhaustive experiments are conducted to show that our method goes beyond all state-of-the-art methods without any tricks. Compared to all methods that do not use additional data for training, our models achieve 67.3% and 41.5% robust accuracy on CIFAR-10 and CIFAR-100 (improving upon the state-of-the-art by +7.23% and +9.07%). We also outperform methods using a training set with a far larger scale than ours. All our models and codes are available online at https://github.com/huangd1999/BORT.
