Case Count Metric for Comparative Analysis of Entity Resolution Results

2026-01-06

arXiv.org

AI Summary

This work addresses the critical challenge of effectively comparing structural differences between two entity resolution (ER) clustering results in the absence of ground-truth labels. It proposes the Case Count Metric System (CCMS), which introduces and operationalizes, for the first time, a quantitative framework for four types of cluster transformations—preservation, merging, splitting, and overlapping—without requiring labeled data. By leveraging a cluster-set transformation analysis algorithm, CCMS enables fine-grained, unsupervised comparison of ER outcomes. Integrated with interactive analysis and visualization capabilities, the system has been successfully deployed in both academic and industrial settings, significantly enhancing the interpretability and efficiency of ER method evaluation and tuning.

Technology Category

• Machine Learning: Clustering
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This paper describes a new process and software system, the Case Count Metric System (CCMS), for systematically comparing and analyzing the outcomes of two different ER clustering processes acting on the same dataset when the true linking (labeling) is not known. The CCMS produces a set of counts that describe how the clusters produced by the first process are transformed by the second process based on four possible transformation scenarios. The transformations are that a cluster formed in the first process either remains unchanged, merges into a larger cluster, is partitioned into smaller clusters, or otherwise overlaps with multiple clusters formed in the second process. The CCMS produces a count for each of these cases, accounting for every cluster formed in the first process. In addition, when run in analysis mode, the CCMS program can assist the user in evaluating these changes by displaying the details for all changes or only for certain types of changes. The paper includes a detailed description of the CCMS process and program and examples of how the CCMS has been applied in university and industry research.

Problem

• Entity Resolution
• Clustering Comparison
• Case Count Metric
• Unlabeled Data
• Result Analysis

Innovation

• Case Count Metric
• Entity Resolution
• Cluster Comparison
• Unsupervised Evaluation
• Clustering Transformation