Benchmarking Image Perturbations for Testing Automated Driving Assistance Systems

2025-01-21

AI Summary

This study addresses the insufficient robustness of deep neural networks in Advanced Driver Assistance Systems (ADAS) against image perturbations—such as noise and illumination variations—and their weak cross-domain generalization capability. To this end, we introduce the first comprehensive, ADAS-specific image perturbation benchmark, encompassing 38 real-world perturbation types. We further propose a perturbation sensitivity quantification framework enabling both component-level and system-level failure analysis. Additionally, we integrate perturbation-driven data augmentation with online continual learning to establish a novel collaborative generalization paradigm. Experimental results demonstrate that all perturbations effectively expose model robustness deficiencies. Our approach improves average detection and segmentation accuracy by 23.6% on unseen scenarios and reduces system failure rate by 41.2%, significantly enhancing ADAS environmental adaptability and operational reliability.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Advanced Driver Assistance Systems (ADAS) based on deep neural networks (DNNs) are widely used in autonomous vehicles for critical perception tasks such as object detection, semantic segmentation, and lane recognition. However, these systems are highly sensitive to input variations, such as noise and changes in lighting, which can compromise their effectiveness and potentially lead to safety-critical failures. This study offers a comprehensive empirical evaluation of image perturbations, techniques commonly used to assess the robustness of DNNs, to validate and improve the robustness and generalization of ADAS perception systems. We first conducted a systematic review of the literature, identifying 38 categories of perturbations. Next, we evaluated their effectiveness in revealing failures in two different ADAS, both at the component and at the system level. Finally, we explored the use of perturbation-based data augmentation and continuous learning strategies to improve ADAS adaptation to new operational design domains. Our results demonstrate that all categories of image perturbations successfully expose robustness issues in ADAS and that the use of dataset augmentation and continuous learning significantly improves ADAS performance in novel, unseen environments.

Problem

• Robustness
• Depth Neural Networks
• Autonomous Driving

Innovation

• Robustness Improvement
• Continual Learning
• Enhanced Dataset