attack evaluation

Inconsistent and unreliable adversarial robustness evaluations arise from model mismatch, non-verifiable implementations, and unequal computational budgets. To address these issues, this paper introduces AttackBench—a standardized benchmarking framework. AttackBench unifies evaluation using gradient-based attacks, a curated set of standard models, and fully reproducible implementations; it further proposes a novel optimality-based metric and strictly controls experimental conditions to ensure fair comparisons. The framework enables trustworthy ranking of mainstream attack methods, systematically identifies sources of bias in existing evaluations, and significantly improves the reproducibility and credibility of robustness verification. Its modular architecture supports continuous extension and benchmark updates, providing a reliable, open evaluation infrastructure for adversarial robustness research.

Existing black-box adversarial attack research lacks a unified, reproducible evaluation benchmark, hindering progress tracking and rigorous technical analysis. To address this, we introduce the first comprehensive benchmark platform designed specifically for realistic, query-limited, and model-inaccessible scenarios. It systematically integrates 25 query-based and 30 transfer-based attack algorithms, evaluated on CIFAR-10 and an ImageNet subset, with full PyTorch implementation supporting gradient estimation, surrogate model training, feature-space transfer, and query optimization. The platform provides a modular codebase, 14,106 cross-model/dataset evaluations, and deep attribution analysis tools. Experimental results reveal fundamental trade-offs among attack success rate, query complexity, and cross-architecture generalization. Our benchmark establishes a reproducible, extensible standard for robustness evaluation—serving as critical infrastructure for advancing black-box adversarial machine learning research.

To address the lack of comprehensive evaluation for black-box adversarial attacks in realistic settings—where only query access is available and robustness to JPEG compression, evasion of automated detectors, and human imperceptibility must all be simultaneously ensured—this paper proposes the first unified quantitative framework for assessing “triple stealthiness.” We introduce ECLIPSE, a novel method integrating Gaussian-blur-based gradient estimation, local surrogate modeling, and multi-objective black-box optimization to jointly optimize these three stealth dimensions. On standard image classification benchmarks, ECLIPSE substantially outperforms state-of-the-art methods: it maintains high attack success rates while reducing post-JPEG accuracy degradation by 37%, lowering detection rates by mainstream defenses by 52%, and driving human subject identification rates down to near-chance level (≈51%). This work delivers the first empirically validated, end-to-end optimization of triple stealthiness under realistic constraints.

This paper identifies a novel security threat in federated learning—Adversarial Robustness Unhardening (ARU): malicious clients inject stealthy backdoors via gradient manipulation to actively degrade the global model’s robustness against evasion attacks. Unlike conventional backdoor attacks that compromise functional correctness, ARU explicitly targets robustness itself—formally characterizing and empirically demonstrating that adversarial robustness can be *reversibly degraded* through distributed training. We propose a dynamic robustness evaluation framework and validate ARU on CIFAR-10 and FMNIST, showing up to 42.7% degradation in robustness of adversarially trained models. Critically, standard robust aggregation mechanisms—including Krum and Trimmed Mean—fail entirely against ARU, exposing a fundamental security gap in federated learning’s robustness guarantees.

Existing adversarial attack evaluations suffer from optimistic bias and irreproducibility due to inconsistent perturbation budgets and non-uniform benchmarks. Method: We propose the first fair and reproducible gradient-based attack evaluation framework, introducing an optimality metric based on multi-attack ensemble estimation, enabling standardized cross-algorithm and cross-model-library assessment under strictly fixed forward/backward query budgets. Contribution/Results: We systematically evaluate over 800 attack configurations on CIFAR-10 and ImageNet, covering more than 100 mainstream implementations. Results reveal that only a few methods exhibit consistent superiority across diverse settings. To foster transparency and rigor, we open-source a benchmarking platform and a dynamic leaderboard—establishing a reliable infrastructure for adversarial robustness research.

This study addresses the critical gap between theory and practice in AI-driven cyberattack prediction, focusing on outdated datasets, limited attack coverage, insufficient model interpretability, weak adversarial robustness, and privacy-ethical risks. Through a systematic review of over 150 benchmark datasets and more than 200 studies, the work introduces a novel multidimensional gap assessment framework based on detection impact, implementation cost, and remediation time to prioritize these challenges. The analysis identifies dataset obsolescence and adversarial robustness as the highest-priority issues, while highlighting interpretability as a cost-effective entry point in resource-constrained settings. Furthermore, the study proposes a tripartite classification of dataset quality—production-ready, research-only, and unusable—alongside a corresponding deployment roadmap, significantly enhancing the practical feasibility and robustness of AI-based cybersecurity systems.

This work addresses robustness challenges of deep learning models in safety-critical applications, tackling three adversarial threats: (1) adversarial examples in computer vision, (2) out-of-distribution generalization (i.e., domain generalization), and (3) jailbreaking attacks against large language models (LLMs). We propose a unified robustness enhancement framework comprising: (i) a certifiably robust defense against adversarial perturbations; (ii) a cross-domain robust training paradigm grounded in out-of-distribution generalization and invariant representation learning; and (iii) an LLM jailbreaking defense integrating formal verification with prompt-attack modeling. Our approach synergistically combines adversarial training, invariance regularization, verification-driven optimization, and controllable decoding. Evaluated on medical image analysis, molecular structure recognition, and standard image classification benchmarks, it achieves state-of-the-art generalization performance. Moreover, it significantly improves jailbreaking resistance across multiple open-source LLMs, demonstrating effectiveness and scalability in multimodal and multi-task settings.

Machine learning–based intrusion detection systems (ML-IDS) exhibit insufficient robustness against black-box adversarial attacks, where attackers manipulate inputs without access to model internals. Method: We propose a lightweight, proactive, and adaptive feature poisoning defense that operates without knowledge of model parameters or training data. Leveraging behavior-aware, context-adaptive network traffic profiling—integrated with change-point detection and dynamic scaling perturbations—it injects imperceptible, targeted perturbations into attacker-critical features. This disrupts the adversary’s attack loop, whether based on binary outputs or behavioral feedback. Contribution/Results: The approach is attack-agnostic, deployment-transparent, and preserves original IDS accuracy. Extensive experiments demonstrate significant reductions in attack success rates across realistic black-box scenarios—including silent probing, transfer attacks, and decision-boundary attacks—while maintaining high generalizability and strong robustness.

Traditional penetration testing struggles to evaluate security risks in AI systems arising from violations of behavioral objectives without breaching underlying infrastructure. This work proposes the first formal definition of AI penetration testing, reframing it as an objective-driven behavioral security assessment. The approach involves identifying operational objectives, mapping AI-driven behaviors, analyzing adversarial attack surfaces—such as prompt injection, data poisoning, and sensor manipulation—establishing criteria for behavioral failure, and conducting scenario-based red-teaming exercises. By integrating threat modeling, behavior mapping, and evidentiary chain construction, the framework demonstrates its efficacy and novelty in a case study involving an AI-powered Security Operations Center assistant, successfully uncovering attack pathways that violate system objectives through behavioral manipulation alone, without requiring infrastructure compromise.

Existing machine learning defense mechanisms primarily focus on the attacks themselves and struggle to identify the attackers, thereby limiting the effectiveness of system-level mitigation strategies. This work proposes the first domain-agnostic framework that shifts the defensive perspective from the attack to the attacker by modeling adversarial behavior and leveraging probabilistic inference to infer attacker characteristics without prior knowledge. Theoretical analysis shows that while attackers cannot be uniquely identified, their attributes can be characterized probabilistically. The framework is applicable across diverse learning models and attack scenarios. Experimental results demonstrate that it not only enhances the precision of exogenous mitigation strategies but also improves the performance of endogenous defense mechanisms such as adversarial regularization.