Criminal Justice Risk Assessments with Unreliable Class Assignments

2026-09-18

AI Summary

论文针对刑事司法风险评估中分类不可靠的问题，提出使用Mondrian保形预测集方法来提高分类可靠性并提供有效的预测不确定性估计。

Abstract

Quantitative risk assessments have been used for decades to help inform criminal justice decisions. A set of predictors and a categorical response variable are used to train a statistical classifier. The trained classifier can be employed to compute risk scores for a new, unlabeled case for which a forecast is needed, one risk score for each response variable class. In the spirit of Bayes classifiers, the class with the largest risk score conventionally becomes the forecasted outcome label for that case. The largest and next largest risk scores can be very similar or very different. However, when they are much the same, the outcome class is essentially being assigned by a coin flip; the classification is unreliable. Forecasting error becomes more likely, and concerns about fairness can arise. This paper shows how Mondrian conformal prediction sets can increase classification reliability when the classifier is indecisive and provide valid estimates of forecast uncertainty. Risk forecasts for a large dataset of offenders on probation are used to illustrate the issues.

Problem

• Criminal Justice
• Risk Assessments
• Unreliable Class Assignments
• Forecasting Error
• Fairness

Innovation

• Mondrian Conformal Prediction Sets
• Classification Reliability
• Forecast Uncertainty