Signal-based Model Access Risk Analysis for AI System Operations Security

2026-07-17

AI Summary

This work addresses a critical limitation in existing black-box attack taxonomies, which overlook variations in output signals from deployed systems, leading to inaccurate security risk assessments. To overcome this, the paper introduces SMART (Signal-oriented Model Access Risk Taxonomy), a fine-grained framework that categorizes attack surfaces based on the richness of information signals returned by real-world deployment interfaces, thereby transcending the traditional binary distinction between white-box and black-box settings. By integrating adversarial example attacks, information signal modeling, and threat modeling, SMART establishes a structured mapping between attacker capabilities and observable output signals. This enables practitioners to conduct actionable, systematic risk evaluations for the secure deployment and procurement of AI systems.

Abstract

Artificial intelligence (AI) systems are now ubiquitous across domains such as security, finance, healthcare, consumer technology, and large-scale cloud services, where they process massive volumes of data and make consequential decisions daily. This widespread adoption has created a broad attack surface through which adversaries can manipulate, evade, extract information from, or otherwise subvert deployed models. Depending on system design and exposure, attackers may have very different forms of access: some observe only final decisions, while others receive confidence scores, intermediate representations, or even full model parameters. While previous surveys typically organize evasion attacks into white-box, gray-box, and black-box categories based on the attacker's knowledge of model internals (architecture, parameters, gradients), this taxonomy often conflates different deployment scenarios that provide vastly different output signals, all labeled as ``black-box'' despite enabling fundamentally different attack strategies. Understanding how evasion attack strategies adapt to the specific information signals returned by deployed systems is critical for organizations making procurement and deployment decisions. To address this gap, we introduce the Signal-based Model Access Risk Taxonomy (SMART), a deployment-oriented framework that classifies attacker access according to the nature and richness of the information signals available from deployed AI systems. Using this taxonomy, we provide a structured overview of evasion attacks across progressively richer levels of information exposure, highlighting how deployment interfaces influence attack capabilities and informing more secure AI deployment and procurement decisions.

Problem

• evasion attacks
• model access
• black-box
• deployment security
• information signals

Innovation

• Signal-based taxonomy
• Model access risk
• Evasion attacks
• AI deployment security
• Information signals