🤖 AI Summary
This paper addresses the challenge of quantitatively assessing privacy risks—specifically identity and attribute disclosure—in practical synthetic data applications. We propose two interpretable, comparable, percentage-based metrics: RepU (Representation Uniqueness), measuring identity disclosure risk relative to original data, and DiSCO (Distributional Similarity Consistency), measuring attribute disclosure risk. We introduce the first well-defined, reproducible dual-metric evaluation framework, innovatively integrating key matching with target-variable consistency testing, augmented by statistical counting and adaptive thresholding. Crucially, the framework automatically filters out spurious disclosures attributable to domain-specific prior knowledge, substantially reducing false positives. Implemented as an extension to the R package *synthpop*, the method is empirically validated on real-world datasets, demonstrating robustness and practical utility for informed synthetic data release decisions.
📝 Abstract
This paper explains how the synthpop package for R has been extended to include functions to calculate measures of identity and attribute disclosure risk for synthetic data that measure risks for the records used to create the synthetic data. The basic function, disclosure, calculates identity disclosure for a set of quasi-identifiers (keys) and attribute disclosure for one variable specified as a target from the same set of keys. The second function, disclosure.summary, is a wrapper for the first and presents summary results for a set of targets. This short paper explains the measures of disclosure risk and documents how they are calculated. We recommend two measures: $RepU$ (replicated uniques) for identity disclosure and $DiSCO$ (Disclosive in Synthetic Correct Original) for attribute disclosure. Both are expressed a % of the original records and each can be compared to similar measures calculated from the original data. Experience with using the functions on real data found that some apparent disclosures could be identified as coming from relationships in the data that would be expected to be known to anyone familiar with its features. We flag cases when this seems to have occurred and provide means of excluding them.