Evaluating Explainability in Safety-Critical ATR Systems: Limitations of Post-Hoc Methods and Paths Toward Robust XAI

2026-05-07

AI Summary

This work addresses the critical need for reliable, verifiable, and robust explainability in safety-critical automatic target recognition (ATR) systems, which demand more than high predictive performance. The authors propose a comprehensive XAI evaluation framework tailored to safety-critical contexts, systematically assessing mainstream post-hoc explanation methods—including saliency maps, attention mechanisms, and surrogate models—along four dimensions: interpretability, robustness, resistance to manipulation, and suitability for formal verification. Their analysis reveals that current approaches commonly suffer from spurious explanations, instability under perturbations, and a tendency to induce unwarranted user trust, rendering them inadequate for ATR assurance requirements. In response, the paper advocates a paradigm shift toward causally grounded, physics-informed explainability, laying the theoretical and technical foundation for next-generation XAI capable of supporting system-level safety guarantees.

Abstract

Explainable Artificial Intelligence (XAI) is increasingly rec ognized as essential for deploying machine learning systems in safety critical environments. In Automatic Target Recognition (ATR), where models operate on image, video, radar, and multisensor data, high pre dictive performance alone is insufficient. Model decisions must also be interpretable, reliable, and suitable for validation. This paper presents a structured evaluation of explainability methods in the context of safety-critical ATR systems: We identify major XAI paradigms, including saliency-based, attention-based, and surrogate ap proaches, as well as recent detection-aware extensions. Based on this, we formalize explainability as an assurance-oriented assessment problem, introduce a taxonomy, and assess these methods with respect to four key dimensions: interpretability, robustness, vulnerability to manipula tion, and suitability for validation and verification. The analysis identifies systematic limitations of current post-hoc explanation methods. In par ticular, we derive critical failure modes such as spurious explanations, instability under perturbations, and overtrust induced by visually con vincing outputs. These findings indicate that widely used XAI techniques may be insufficient for safety-critical deployment. Finally, we discuss implications for ATR systems and outline directions toward more robust, causally grounded, and physically informed explain ability methods. Our results emphasize the need to move beyond visually plausible explanations toward approaches that support reliable decision making and system-level assurance.

Problem

• Explainable Artificial Intelligence
• Automatic Target Recognition
• post-hoc explanation
• safety-critical systems
• model interpretability

Innovation

• Explainable AI (XAI)
• Automatic Target Recognition (ATR)
• post-hoc explanation
• robustness
• assurance-oriented evaluation