🤖 AI Summary
Deterministic algorithms used in U.S. pretrial risk assessment lack strategic optimizability and interpretability, limiting their ability to improve upon existing policies safely and transparently.
Method: We propose an extrapolation-based safe policy learning framework that integrates robust optimization (maximizing minimum expected utility), partial identification theory, and causal inference—enabling safe policy improvement without assuming randomized interventions.
Contribution/Results: To our knowledge, this is the first method to guarantee, with high probability, that a new deterministic policy dominates the current one under observational data, while ensuring statistical safety and interpretability. Evaluated on real-world field experiment data, our approach significantly increases the proportion of defendants in specific subgroups who are safely downgraded to “low risk,” without compromising overall decision safety. This advances responsible deployment of judicial algorithms by providing a principled, auditable, and policy-improving paradigm.
📝 Abstract
Algorithmic recommendations and decisions have become ubiquitous in today's society. Many of these data-driven policies, especially in the realm of public policy, are based on known, deterministic rules to ensure their transparency and interpretability. We examine a particular case of algorithmic pre-trial risk assessments in the US criminal justice system, which provide deterministic classification scores and recommendations to help judges make release decisions. Our goal is to analyze data from a unique field experiment on an algorithmic pre-trial risk assessment to investigate whether the scores and recommendations can be improved. Unfortunately, prior methods for policy learning are not applicable because they require existing policies to be stochastic. We develop a maximin robust optimization approach that partially identifies the expected utility of a policy, and then finds a policy that maximizes the worst-case expected utility. The resulting policy has a statistical safety property, limiting the probability of producing a worse policy than the existing one, under structural assumptions about the outcomes. Our analysis of data from the field experiment shows that we can safely improve certain components of the risk assessment instrument by classifying arrestees as lower risk under a wide range of utility specifications, though the analysis is not informative about several components of the instrument.
