neural network training

This work addresses the fundamental trade-off between standard accuracy and adversarial robustness in supervised learning. Methodologically, it introduces the first architecture-level accuracy–robustness trade-off curve, quantifying the inverse relationship between these objectives across diverse neural network architectures; defines a sensitivity influence function to theoretically characterize the stability of optimal solutions under adversarial perturbations; and reveals—via theoretical analysis of overparameterized linear models—that adversarial training implicitly regularizes model dynamics, interpolating between L₁ (LASSO) and L₂ (ridge regression) behaviors. The approach integrates rigorous theoretical analysis, influence-function-based modeling, and extensive empirical evaluation across fully connected, deep, and varying-width networks. Results consistently validate the existence and structure of the trade-off, providing an interpretable, predictive theoretical foundation for principled neural architecture selection.

Fair evaluation of deep learning training algorithms faces three key challenges: inconsistent termination criteria, high workload sensitivity, and difficulty isolating hyperparameter tuning. This paper introduces AlgoPerf—the first time-oriented, multi-workload training algorithm benchmark—featuring robustness-aware workload variant design and a standardized termination protocol, with hyperparameter tuning rigorously isolated. Evaluated on a unified hardware platform, AlgoPerf employs a diverse multi-task workload suite and a systematic optimizer comparison methodology to enable latency-accuracy co-evaluation across models, datasets, and hardware. Experiments reveal substantial latency disparities among mainstream optimizers, establish reproducible state-of-the-art baselines, and deliver the first quantitative, fair, and engineering-practical evaluation standard for training algorithm improvement.

The lack of standardized benchmarks hinders rigorous evaluation of physically realizable adversarial patch robustness. Method: We introduce ImageNet-Patch—the first large-scale, multi-variant adversarial patch benchmark covering all 1,000 ImageNet classes—and systematically incorporate realistic physical perturbations, including random scaling, placement, lighting, and viewpoint variations. We further propose the first standardized robustness evaluation protocol, enabling fair, quantitative comparison across over 20 mainstream models. Contribution/Results: Experiments reveal that state-of-the-art models suffer an average accuracy drop exceeding 40% under patch attacks, exposing critical vulnerabilities. ImageNet-Patch fills a key gap in the field, providing a unified, reproducible infrastructure for developing and evaluating robust training methods against physical-world adversarial patches.

Existing certified training algorithms suffer from inconsistent evaluation protocols and suboptimal hyperparameter tuning, leading to incomparable performance claims and unreliable SOTA conclusions. Method: We introduce CTBENCH—the first unified benchmark for certified training—enabling fair, cross-algorithm evaluation of mainstream methods (e.g., IBP, CROWN-IBP, DeepPoly) under a standardized training pipeline, consistent ℓ∞/ℓ2 certification framework, and systematic hyperparameter optimization (grid search + Bayesian optimization). Contribution/Results: Our evaluation reveals that most recently proposed algorithms are substantially overestimated in prior work; after baseline enhancement, their relative improvements drop by over 40% on average. Crucially, all methods achieve significantly higher certified accuracy on CTBENCH than reported in their original papers. This work establishes a reproducible, extensible standard for evaluating certified training, redefining both the robustness training baseline and the SOTA landscape.

Existing neural network verifiers lack “source-grounded ground truth”—i.e., labels for hard instances that are truly unverifiable and resistant to counterexample discovery—making it difficult to validate claims of solving challenging cases. Method: We introduce the first verifier benchmark designed for source-grounded evaluation, proposing implicit counterexample injection training to embed hidden, semantically valid counterexamples imperceptible to standard adversarial attacks. Our framework enables controllable generation of hard-to-verify instances across architectures, activation functions, input dimensions, and perturbation radii. By integrating adversarial robust training, gradient masking mitigation, and multi-dimensional parametric synthesis, we systematically construct instances exposing fundamental soundness flaws. Contribution/Results: The benchmark uncovers both real and synthetic soundness bugs in multiple state-of-the-art verifiers. All code and datasets are publicly released, establishing a standardized foundation for rigorous, reproducible verifier reliability assessment.

This work addresses the vulnerability of current object detection models to adversarial attacks and the absence of a standardized benchmark for fair comparison. We propose the first unified evaluation framework tailored to digital, non-patch adversarial attacks, which decouples localization and classification errors, introduces multi-dimensional perceptual metrics to quantify perturbation cost, and systematically assesses attack transferability and defense strategies. Our experiments reveal two key findings: first, existing attacks exhibit notably limited transferability between CNN- and Vision Transformer–based architectures; second, adversarial training using a hybrid approach that combines high-magnitude, multi-target attacks substantially outperforms training with single-attack variants, leading to significantly improved model robustness.

To address the low training efficiency, poor generalization, and strong hyperparameter sensitivity of neural networks across varying scales, this paper proposes a scale-invariant adaptive optimization framework. The method unifies adaptive optimization, second-order information approximation, learning-rate scaling invariance, and gradient compression, thereby decoupling optimization from model size and hardware configuration. Its core innovation lies in a scale-robust update paradigm that ensures stable optimization dynamics under variations in parameter count, batch size, and device count. Extensive experiments across diverse architectures—including MLPs, CNNs, and Transformers—and benchmarks—including CIFAR-10/100, ImageNet, and WikiText—demonstrate that the framework achieves 1.3–2.1× speedup over baseline optimizers, improved convergence stability, significantly reduced hyperparameter sensitivity, and eliminates the need for scale-specific hyperparameter tuning.

This work investigates whether adversarial training in nonlinear models can be reduced to a regularization problem. Focusing on two-layer neural networks, the study provides the first theoretical proof that adversarial risk cannot be equivalent to any weakly data-dependent regularized risk. Empirical evidence further supports this finding on deep architectures such as Wide-ResNet. By integrating theoretical reduction with experimental analysis, the research reveals a fundamental distinction between adversarial robustness and conventional regularization, establishing a clear boundary between the two paradigms. Consequently, it demonstrates that efficient approximation methods proven effective for linear models do not directly generalize to nonlinear neural networks.

This study addresses the critical gap between theory and practice in AI-driven cyberattack prediction, focusing on outdated datasets, limited attack coverage, insufficient model interpretability, weak adversarial robustness, and privacy-ethical risks. Through a systematic review of over 150 benchmark datasets and more than 200 studies, the work introduces a novel multidimensional gap assessment framework based on detection impact, implementation cost, and remediation time to prioritize these challenges. The analysis identifies dataset obsolescence and adversarial robustness as the highest-priority issues, while highlighting interpretability as a cost-effective entry point in resource-constrained settings. Furthermore, the study proposes a tripartite classification of dataset quality—production-ready, research-only, and unusable—alongside a corresponding deployment roadmap, significantly enhancing the practical feasibility and robustness of AI-based cybersecurity systems.

This work addresses the lack of systematic quantification of clean-accuracy degradation in adversarially robust models, which hinders realistic assessment of their deployment cost. The authors propose VanillaBench, the first benchmark that systematically computes the clean accuracy gap between 186 adversarially trained models and their corresponding standard (vanilla) counterparts by leveraging RobustBench and Papers with Code datasets through multi-strategy controls—including temporal alignment and architecture matching. Results reveal that robust models underperform the best contemporary vanilla models by 4.0–29.5 percentage points on clean accuracy; even after strict architecture matching, the gap remains substantial at 3.5–17.5 points. These findings expose a severely underestimated robustness–accuracy trade-off and advocate for incorporating vanilla-reference accuracy gaps into standard robustness evaluation protocols.