AI for Scaling Legal Reform: Mapping and Redacting Racial Covenants in Santa Clara County

2025-02-12

AI Summary

Racial restrictive covenants—unenforceable since 1948 yet persisting in historical U.S. property deeds—remain difficult to identify and remove at scale. Method: This paper introduces the first lightweight, open-source large language model fine-tuning framework designed specifically for legal reform, trained on 24 million property deeds from Santa Clara County. It integrates legal text structural analysis, natural language processing, and geospatial mapping to achieve high-precision covenant localization and detection. Contributions/Results: Our model outperforms commercial closed-source systems in detection accuracy while costing less than 2% of their deployment expense and saving 86,500 person-hours. It reveals, for the first time, spatiotemporal distribution patterns of racial covenants and identifies developer-driven enforcement as a dominant mechanism. We confirm that one-quarter of county properties were bound by such covenants in 1950, establishing a reproducible technical paradigm for redress of systemic housing discrimination and judicial practice.

Technology Category

• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models
• Machine Learning: Feature Construction/Reformulation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Many jurisdictions have moved to identify and strike these provisions, including California, which mandated in 2021 that all counties implement such a process. Yet the scale can be overwhelming, with Santa Clara County (SCC) alone having over 24 million property deed documents, making purely manual review infeasible. We present a novel approach to addressing this pressing issue, developed through a partnership with the SCC Clerk-Recorder's Office. First, we leverage an open large language model, fine-tuned to detect racial covenants with high precision and recall. We estimate that this system reduces manual efforts by 86,500 person hours and costs less than 2% of the cost for a comparable off-the-shelf closed model. Second, we illustrate the County's integration of this model into responsible operational practice, including legal review and the creation of a historical registry, and release our model to assist the hundreds of jurisdictions engaged in similar efforts. Finally, our results reveal distinct periods of utilization of racial covenants, sharp geographic clustering, and the disproportionate role of a small number of developers in maintaining housing discrimination. We estimate that by 1950, one in four properties across the County were subject to racial covenants.

Problem

• AI detects racial covenants in property deeds efficiently.
• Reduces manual effort by 86,500 hours, cutting costs significantly.
• Identifies historical patterns and geographic clustering of racial covenants.

Innovation

• Open large language model for precise racial covenant detection
• Integration into legal review and historical registry creation
• Significant reduction in manual effort and operational costs