We Need Explanation Cards to Connect Explanation Algorithms to the Real World

2026-06-15

AI Summary

This study addresses the challenge that existing algorithmic explanations are often poorly understood and misapplied by non-expert users due to semantic ambiguity and insufficient contextual information, leading to a disconnect between explanations and actual decision-making. To bridge this gap, the authors propose an “Explanation Card” framework that augments widely used interpretability methods—such as SHAP and counterfactual explanations—with structured metadata specifying their applicability boundaries, robustness properties, and user-oriented interpretation guidance. By shifting explanatory responsibility from end users to explanation providers, this approach enhances the practical utility and regulatory compliance of model explanations, aligning with the transparency requirements of the EU AI Act. Empirical evaluations demonstrate that Explanation Cards significantly improve users’ comprehension accuracy of complex model explanations and effectively flag scenarios where explanations are unreliable, thereby facilitating trustworthy real-world deployment of algorithmic systems.

Abstract

Algorithmic explanations are intended to help stakeholders understand opaque algorithmic decisions, but in practice, they often fall short. First, the meaning of algorithmic explanations is often not what one might intuitively expect, so expert knowledge is required to interpret them correctly. Second, recent work has shown that popular explanation algorithms are uninformative about the behavior of complex decision functions. Together, these issues create a gap between what explanations appear to convey and what they actually provide. In this work, we propose Explanation Cards for Explanation Algorithms, which augment standard explanations with complementary information about robustness and validity, as well as clear instructions for interpretation. The complementary information can render otherwise uninformative explanations practically useful, while also helping to detect cases where they are not. Importantly, the interpretation instructions in explanation cards shift responsibility from users to providers: Rather than expecting users to recognize what can and cannot be concluded from an explanation, providers must make this explicit upfront. Using counterfactual explanations and SHAP as examples, we demonstrate how providers can construct explanation cards and that these cards provide users with the guidance needed for sound interpretation. We further argue that explanation cards offer a practical means of operationalising the explainability provisions of the EU AI Act. Overall, explanation cards are a significant step toward making explanation algorithms fit for real-world use cases.

Problem

• algorithmic explanations
• interpretability
• explanation validity
• real-world deployment
• explainable AI

Innovation

• Explanation Cards
• Algorithmic Explanation
• Interpretability
• SHAP
• Counterfactual Explanations