Fast Object Removal Attacks on Safety-Critical Video-based Perception Systems

2026-08-03

AI Summary

This work addresses the vulnerability of video-based traffic safety perception systems to object removal attacks by proposing the first end-to-end near-real-time adversarial framework. The approach operates through four stages—object localization, temporally consistent patch retrieval, context-aware fusion, and frame reconstruction—achieving high visual fidelity (PSNR > 40 dB, SSIM > 0.996) while substantially degrading detection performance. Evaluated on the SC-CVT platform, the method attains a 94.48% frame-level attack success rate, reducing object detection rates by up to 97.59%, with per-frame processing times ranging from 0.074 to 0.172 seconds. The framework demonstrates strong efficiency, stealthiness, and temporal consistency, and further exposes critical limitations in current tampering detection models against such sophisticated attacks.

Abstract

By leveraging data from video-based perception systems, intelligent transportation systems (ITS) support safety-critical applications that improve road safety. However, adversaries may manipulate video frames to compromise downstream perception modules, causing failures in safety-critical functions and increasing risks to vulnerable road users. This paper presents a novel attack model and an end-to-end framework for near-real-time targeted object removal attack on a video-based safety-critical system. The end-to-end attack pipeline consists of four stages: localizing targets in each frame, retrieving coherent patches from earlier frames, blending them using context-aware alpha compositing, and reconstructing attacked frames. Experiments at an intersection on the South Carolina Connected Vehicle Testbed (SC-CVT) show that reconstructed frames have high global similarity to the originals, with frame-level Peak Signal to Noise Ratio (PSNR) above 40 dB and Structural Similarity Index Measure (SSIM) above 0.996. Using the YOLO-based detector, the attack reduces object detections by up to 97.59% and achieves a frame-level attack success rate of 94.48%. Across the evaluated detectors and frame resolutions, the mean execution time ranges from 0.074 to 0.172 seconds per frame on GPU hardware, indicating near-real-time performance in testing. The forensic evaluation using several pretrained tamper-detection models shows limited ability to distinguish reconstructed from authentic frames. The findings suggest that video-based perception is vulnerable to stealthy object removal attacks that can degrade the performance of safety-critical applications by reducing object detectability. These findings can help develop mitigation strategies against adversarial object removal attacks that threaten safety-critical applications, such as vision-based pedestrian safety systems.

Problem

• object removal attack
• video-based perception
• safety-critical systems
• adversarial attack
• intelligent transportation systems

Innovation

• object removal attack
• video-based perception
• adversarial attack
• real-time performance
• context-aware blending