🤖 AI Summary
This study addresses the challenge in credit risk model validation where significant declines in the Kolmogorov–Smirnov (KS) statistic often lead to subjective judgments due to the absence of standardized attribution methods, undermining governance consistency and transparency. To resolve this, the authors propose a structured counterfactual diagnostic framework that sequentially attributes performance deterioration—under gated conditions—to sampling variability, portfolio composition shifts, covariate shift, or model drift. Integrating counterfactual reasoning, KS decomposition, and simulation-based testing, this approach delivers the first interpretable, governance-oriented system for systematically diagnosing KS declines. Empirical results demonstrate that, compared to conventional threshold-based reviews, the framework yields diagnostics with greater business relevance and regulatory value, substantially enhancing the rigor, consistency, and defensibility of model validation practices.
📝 Abstract
The Kolmogorov-Smirnov (KS) statistic is widely used in credit risk model monitoring and validation to assess discriminatory power. In practice, a material decline in KS often triggers governance review and requires validation teams to identify the breach source and the potential business risk. However, such diagnosis is frequently conducted on an ad hoc basis, relying on the judgment of individual validators rather than a standardized analytical framework. This paper proposes a counterfactual diagnostic framework for explaining KS deterioration in credit risk model validation. The framework sequentially attributes observed KS decline to sampling variability, portfolio composition change, covariate shift, and residual deterioration consistent with model drift, with explicit gateway conditions governing escalation at each stage. Simulation experiments demonstrate that the proposed approach provides more interpretable and governance-relevant explanations than threshold-based review alone, and contributes to more consistent, transparent, and defensible performance-breach assessment in credit risk model validation.