🤖 AI Summary
In credit risk management, binary classifiers often suffer from miscalibrated probability estimates due to target prior distribution shift. To address this, we propose two novel calibration methods: CSPD (Calibration via Separable Parametric Modeling of Covariate and Posterior Drift) and ROC-based QMM (Quantile Moment Matching under ROC constraints). First, we systematically uncover and formalize the distribution-invariance assumption implicitly encoded in the AUC metric, integrating it into the calibration framework design. CSPD enables interpretable, parametric calibration by decoupling covariate shift from posterior shift. QMM delivers conservative estimates of concave objectives—such as credit risk-weighted metrics—under ROC curve constraints. Experiments on real-world financial datasets demonstrate that QMM significantly outperforms standard baselines—including Platt scaling and isotonic regression—while maintaining high robustness and practicality under unknown prior distributions.
📝 Abstract
Recalibration of binary probabilistic classifiers to a target prior probability is an important task in areas like credit risk management. We analyse methods for recalibration from a distribution shift perspective. Distribution shift assumptions linked to the area under the curve (AUC) of a probabilistic classifier are found to be useful for the design of meaningful recalibration methods. Two new methods called parametric covariate shift with posterior drift (CSPD) and ROC-based quasi moment matching (QMM) are proposed and tested together with some other methods in an example setting. The outcomes of the test suggest that the QMM methods discussed in the paper can provide appropriately conservative results in evaluations with concave functionals like for instance risk weights functions for credit risk.