🤖 AI Summary
This work proposes an unsupervised record linkage method that operates without labeled data by leveraging semantic knowledge from open-source pretrained language models to enable cross-dataset entity matching. Addressing the challenge of simultaneously achieving high accuracy and reliable uncertainty quantification in label-scarce settings, the approach matches or exceeds the performance of supervised methods on multiple benchmarks—including city names, personal names, organizations, and multilingual political party and bibliographic records—despite requiring no human-annotated training data. Notably, it attains supervised-level accuracy under a fully unsupervised setting, supports local deployment without reliance on external APIs, and thereby substantially reduces privacy risks and accessibility barriers. The method completes typical linkage tasks within minutes, demonstrating both efficiency and practicality.
📝 Abstract
Record linkage, the process of matching records that refer to the same entity across datasets, is essential to empirical social science but remains methodologically underdeveloped. Researchers treat it as a preprocessing step, applying ad hoc rules without quantifying the uncertainty that linkage errors introduce into downstream analyses. Existing methods either achieve low accuracy or require substantial labeled training data. I present EnsembleLink, a method that achieves high accuracy without any training labels. EnsembleLink leverages pre-trained language models that have learned semantic relationships (e.g., that"South Ozone Park"is a neighborhood in"New York City"or that"Lutte ouvriere"refers to the Trotskyist"Workers'Struggle"party) from large text corpora. On benchmarks spanning city names, person names, organizations, multilingual political parties, and bibliographic records, EnsembleLink matches or exceeds methods requiring extensive labeling. The method runs locally on open-source models, requiring no external API calls, and completes typical linkage tasks in minutes.