Statistical Adversaries: Natural Backdoor-like Features in Vision Datasets

2026-07-06

AI Summary

This work demonstrates that naturally occurring statistical signals in visual datasets can function as implicit backdoor triggers, enabling manipulation of model predictions without malicious data poisoning. By employing statistical association analysis and conditional control variable methods, the study systematically identifies non-semantic patterns in ImageNet that exhibit strong correlations with specific labels. These patterns are shown to be transferable across diverse model architectures and capable of steering predictions in a controllable manner. The paper introduces the novel concept of “statistical adversarial examples,” revealing that inherent spurious structures in datasets not only serve as sources of bias but also constitute cross-model security vulnerabilities. Notably, such attacks exhibit greater target specificity compared to generic image corruptions.

Abstract

Model-specific adversarial attacks have been extensively studied. We study a different failure mode: naturally occurring statistical signals in vision data that can behave like backdoor-like triggers without being maliciously inserted. We call these signals statistical adversaries. We analyse Imagenet to find patterns that are strongly linked to certain labels. We then use statistical controls to remove random correlations from our candidate signals. Finally, we demonstrate that these signals directly and predictably alter model predictions. These statistical adversaries are more targeted than generic corruptions and transfer across different model architectures. This suggests that some vulnerabilities are driven by dataset structure and distribution rather than a single model's idiosyncrasies. We conclude that ordinary datasets can contain exploitable adversarial surfaces even in the absence of poisoning, and suggest that dataset audits should treat spurious structure not only as a source of bias or interpretability failure, but also as a latent attack surface for vision models.

Problem

• statistical adversaries
• backdoor-like features
• vision datasets
• dataset bias
• adversarial vulnerabilities

Innovation

• statistical adversaries
• natural backdoor
• dataset bias
• transferable attacks
• vision model vulnerabilities