Credit Risk Meets Large Language Models: Building a Risk Indicator from Loan Descriptions in P2P Lending

2024-01-29

Inteligencia Artificial

AI Summary

To address information asymmetry in peer-to-peer (P2P) lending caused by the lack of structured representation in borrower-provided textual descriptions, this paper proposes an interpretable, text-driven credit assessment framework. Methodologically, it introduces fine-tuned BERT to generate fine-grained textual risk scores, which are embedded as structured features into an XGBoost credit model; attribution analysis further reveals how these scores conditionally adjust based on loan purpose. The contributions are twofold: (1) a semantics-aware risk metric integrating borrower traits, fund utilization intent, and linguistic style; and (2) dynamic joint modeling of textual and conventional tabular features. Experiments demonstrate significant improvements in balanced accuracy (+3.2%) and AUC (+4.7%), achieving both enhanced predictive performance and transparent, interpretable decision-making.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Peer-to-peer (P2P) lending connects borrowers and lenders through online platforms but suffers from significant information  asymmetry, as lenders often lack sufficient data to assess borrowers’ creditworthiness. This paper addresses this challenge by leveraging BERT, a Large Language Model (LLM) known for its ability to capture contextual nuances in text, to generate a risk score based on borrowers’ loan descriptions using a dataset from the Lending Club platform. We fine-tune BERT to distinguish between defaulted and non-defaulted loans using the loan descriptions provided by the borrowers. The resulting BERT-generated risk score is then integrated as an additional feature into an XGBoost classifier used at the loan granting stage, where decision-makers have limited information available to guide their decisions. This integration enhances predictive performance, with improvements in balanced accuracy and AUC, highlighting the value of textual features in complementing traditional inputs. Moreover, we find that the  incorporation of the BERT score alters how classification models utilize traditional input variables, with these changes varying by loan purpose. These findings suggest that BERT discerns meaningful patterns in loan descriptions, encompassing borrower-specific features, specific purposes, and linguistic characteristics. However, the inherent opacity of LLMs and their potential biases underscore the need for transparent frameworks to ensure regulatory compliance and foster trust. Overall, this study demonstrates how LLM-derived insights interact with traditional features in credit risk modeling, opening new avenues to enhance the explainability and fairness of these models.

Problem

• Assessing credit risk in P2P lending using loan descriptions
• Integrating BERT-generated risk scores with traditional credit models
• Improving loan default prediction with LLM-enhanced textual features

Innovation

• Fine-tune BERT for loan default prediction
• Integrate BERT score into XGBoost classifier
• Enhance credit risk modeling with text