Hidden Errors in Big Data: The Case of Property Records

2026-07-30

AI Summary

This study addresses systematic errors in real estate broker data that may bias conclusions in economic inequality research. By auditing two major property listing datasets and matching them to verified transaction records, the authors identify pervasive issues—including price misreporting, incomplete coverage, and inconsistent attribute definitions—and quantify their impact on estimates of property tax regressivity. This work provides the first systematic assessment of the magnitude and cross-source consistency of such errors, revealing that 1–2% of transactions exhibit price discrepancies exceeding 5%, while 12–15% suffer from coverage inaccuracies. These data imperfections substantially alter assessments of tax regressivity, with findings demonstrating robustness across three large U.S. counties.

Abstract

Big data are the foundation for an increasing share of academic research and AI models deployed in both the public and private sectors, prompting substantial growth over time in reliance on brokered datasets. Brokered property records, which are ubiquitous in studies of gentrification, inequality, and the property tax in the U.S. and serve as inputs to property valuation models, are one notable example. In this paper, we audit two prominent brokered property datasets, finding errors in these data which bias key measures of economic inequality. First, we document that for 1-2% of matched sales in Cook County, IL, from 2018-2021, broker-provided sale prices differ from ground truth sale prices by more than 5%. Moreover, missing data and conceptual differences in the reporting of deed and property characteristics lead to coverage errors ranging from 12 to 15% of transactions. Second, we show that misreporting is highly consistent between brokers: more often than not, brokers make identical reporting errors for the same transactions. Third, to illustrate the significance of these errors, we measure their impact on estimates of property tax regressivity, finding that they drive significant wedges between estimates depending on the data source. These findings generalize to two other large counties in the U.S., and highlight the crucial importance of open administrative data and transparency from brokers regarding data provenance and lineage.

Problem

• big data errors
• property records
• economic inequality
• data quality
• brokered datasets

Innovation

• data auditing
• brokered property records
• measurement error
• economic inequality
• data provenance