Detectors Learn the Wrong Thing: Shortcut-Resistant Adversarial Training Against Physically Realizable Attacks

2026-07-23

AI Summary

This work addresses a critical limitation in existing adversarial training for object detectors, which inadvertently encourages models to treat adversarial textures as standalone evidence of object presence, leading to high false-positive rates on texture-only inputs and degraded generalization against physical attacks. To mitigate this “texture shortcut” issue—first identified in this study—we propose InsCAT, an instance-level contrastive adversarial training framework. InsCAT leverages Sample-wise Instance Contrastive Alignment (SICA) to align features of positive samples while disentangling negatives, and integrates Robust Online Perturbation Optimization (ROPO) with a Guard coordination mechanism to regularize the feature space and suppress reliance on spurious textures. Experiments demonstrate that InsCAT achieves an average attacked AP of 82.3% on nuScenes, surpassing the strongest baseline by 11.1 points, reduces texture-induced false positives from 46.9% to 7.3%, and attains a physical-test F1 score of 96.6% with a false-positive rate of only 1.8%.

Abstract

AI-enabled visual perception systems are increasingly deployed in intelligent transportation infrastructure and autonomous vehicle related applications. However, physically realizable adversarial appearances pose a significant reliability challenge for these safety-critical systems. Adversarial training is effective, but repeated co-occurrence between adversarial texture and positive person instances can cause detectors to treat the texture itself as evidence of object presence, forming a patch texture shortcut. The detector may then treat texture as evidence for the target, causing false detections on texture-only inputs and weakening cross attack generalisation. We propose InsCAT, an instance-level contrastive adversarial training framework that prevents detectors from using adversarial texture as an independent decision cue. SICA aligns adversarial person features with matched clean features and separates them from texture-only negatives, while ROPO and Guard maintain online attack pressure and coordinate training. We evaluate eight independently generated attack textures on rendered nuScenes, INRIAPerson, printed garments, and three detector families. InsCAT achieves an average attack AP of 82.3% on rendered nuScenes, exceeding the strongest baseline by 11.1 points.Relative to AT-Mix, texture FPR decreases from 46.9% to 7.3%. Physical tests yield an F1 score of 96.6% and an FPR of 1.8%. Consistent gains across separately trained detectors demonstrate applicability across architectures with direct inference. The findings show that robust physical detection depends on preserving target related evidence while preventing adversarial texture from becoming an independent decision cu

Problem

• adversarial training
• physically realizable attacks
• shortcut learning
• object detection
• texture shortcut

Innovation

• adversarial training
• shortcut learning
• instance-level contrastive learning
• physically realizable attacks
• object detection robustness