🤖 AI Summary
This study addresses the failure of regulatory accountability caused by accumulating “interpretability debt” in production AI systems. To this end, it proposes the TRACE governance framework, which introduces a novel Explainability Debt Score (EDS) and a seven-tool architecture. Through DART trajectory analysis, SHIV validation, and FDE feature drift evaluation, the framework enables systematic quantification, tracking, and remediation of interpretability debt. Empirical results demonstrate that TRACE accurately predicts debt evolution trends, reveals explanation asymmetries in high-risk decision-making, and comprehensively aligns with EU AI Act compliance requirements. This work establishes a new paradigm for AI explainability governance, providing methodological support for sustained regulatory compliance in production-grade systems.
📝 Abstract
Production AI systems deployed in high-stakes domains accumulate a governance liability that existing monitoring frameworks fail to detect: the progressive inability to explain individual decisions when regulators, auditors, or affected individuals demand accountability. We introduce TRACE (Transparency, Risk, Accountability, Compliance, and Explainability), a seven-instrument governance framework for measuring, tracking, and remediating Explainability Debt in production AI systems. The foundational instrument, the Explainability Debt Score (EDS), quantifies the proportion of production decisions falling below a governance-defined explainability confidence threshold at any point in time. Complementary instruments include DART (Debt Accumulation Rate Tracker for breach forecasting), SHIV (Scenario Health and Integrity Validator for daily governance), FDE (Feature Drift Evaluator for causal attribution), HVE (Human Validation Engine), AIDE (Audit Intervention Decision Engine), and ZERO (Zero Explainability Risk Optimiser for remediation). Through a twelve-month longitudinal case study of a production fraud detection system processing 50,000 daily financial transactions, achieving 98.46% accuracy and ROC-AUC of 0.9990, we demonstrate that an EDS of 0.23 on audit day was statistically predictable six months in advance using DART trajectory analysis (beta = 0.008/week, R-squared = 0.94, 95% CI: [0.006, 0.010]), and that 78% of Explainability Debt was concentrated in the highest-regulatory-risk decision category (transactions above $10,000), a risk asymmetry completely invisible to system-level metrics. TRACE provides the first quantitative operational architecture for EU AI Act Article 13 compliance in production AI deployment, establishing a new subdiscipline of explanation governance distinct from explanation generation.