An Information-Theoretic Framework for Credit Risk Modeling: Unifying Industry Practice with Statistical Theory for Fair and Interpretable Scorecards

2025-09-11

AI Summary

Widely used metrics in credit risk modeling—Weight of Evidence (WoE), Information Value (IV), and Population Stability Index (PSI)—lack a unified theoretical foundation and are disconnected from information theory. Method: We propose the first unified information-theoretic framework: (i) proving IV equals the Jeffreys divergence between good and bad class distributions; (ii) linking PSI to the Kullback–Leibler (KL) divergence; (iii) deriving delta-method-based standard errors for IV and PSI to enable statistical inference and probabilistic fairness constraints; (iv) automating binning via depth-1 XGBoost leaf nodes; and (v) evaluating three encoding schemes (one-hot, WoE, constrained XGBoost) under mixed-integer programming to compute the accuracy–fairness Pareto frontier. Results: Experiments show stable AUC (0.82–0.84) across encodings, confirming that information-theoretic binning—not encoding choice—drives performance. Our framework enables interpretable, quantifiable, and statistically testable joint optimization of accuracy and fairness.

Technology Category

• Machine Learning: Information Theory
• Knowledge Representation and Reasoning: Preferences
• Reasoning under Uncertainty: Decision/Utility Theory

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Credit risk modeling relies extensively on Weight of Evidence (WoE) and Information Value (IV) for feature engineering, and Population Stability Index (PSI) for drift monitoring, yet their theoretical foundations remain disconnected. We establish a unified information-theoretic framework revealing these industry-standard metrics as instances of classical information divergences. Specifically, we prove that IV exactly equals PSI (Jeffreys divergence) computed between good and bad credit outcomes over identical bins. Through the delta method applied to WoE transformations, we derive standard errors for IV and PSI, enabling formal hypothesis testing and probabilistic fairness constraints for the first time. We formalize credit modeling's inherent performance-fairness trade-off as maximizing IV for predictive power while minimizing IV for protected attributes. Using automated binning with depth-1 XGBoost stumps, we compare three encoding strategies: logistic regression with one-hot encoding, WoE transformation, and constrained XGBoost. All methods achieve comparable predictive performance (AUC 0.82-0.84), demonstrating that principled, information-theoretic binning outweighs encoding choice. Mixed-integer programming traces Pareto-efficient solutions along the performance-fairness frontier with uncertainty quantification. This framework bridges theory and practice, providing the first rigorous statistical foundation for widely-used credit risk metrics while offering principled tools for balancing accuracy and fairness in regulated environments.

Problem

• Unifying industry credit risk metrics with information theory foundations
• Enabling hypothesis testing for fairness constraints in scorecards
• Balancing predictive performance and fairness in regulated environments

Innovation

• Unified information-theoretic framework for credit metrics
• Derived standard errors enabling hypothesis testing
• Mixed-integer programming for performance-fairness trade-off