MultiGraSCCo: A Multilingual Anonymization Benchmark with Annotations of Personal Identifiers

2026-03-09

AI Summary

This work addresses the scarcity of high-quality training and evaluation data for anonymization systems in multilingual and low-resource settings, a challenge exacerbated by privacy regulations limiting access to sensitive patient data. To bridge this gap, the authors introduce the first cross-lingual anonymization benchmark dataset spanning ten languages, comprising over 2,500 annotated personally identifiable information (PII) instances. The dataset is constructed by transferring real or synthetic data and their annotations from high-resource languages to target languages via neural machine translation, augmented with culturally adapted named entity handling and cross-lingual annotation alignment to preserve semantic and contextual fidelity. Expert medical evaluation confirms the dataset’s validity, demonstrating its utility for developing anonymization models, training annotators, and enabling cross-institutional system validation in low-resource linguistic contexts.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Privacy
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Accessing sensitive patient data for machine learning is challenging due to privacy concerns. Datasets with annotations of personally identifiable information are crucial for developing and testing anonymization systems to enable safe data sharing that complies with privacy regulations. Since accessing real patient data is a bottleneck, synthetic data offers an efficient solution for data scarcity, bypassing privacy regulations that apply to real data. Moreover, neural machine translation can help to create high-quality data for low-resource languages by translating validated real or synthetic data from a high-resource language. In this work, we create a multilingual anonymization benchmark in ten languages, using a machine translation methodology that preserves the original annotations and renders names of cities and people in a culturally and contextually appropriate form in each target language. Our evaluation study with medical professionals confirms the quality of the translations, both in general and with respect to the translation and adaptation of personal information. Our benchmark with over 2,500 annotations of personal information can be used in many applications, including training annotators, validating annotations across institutions without legal complications, and helping improve the performance of automatic personal information detection. We make our benchmark and annotation guidelines available for further research.

Problem

• anonymization
• personally identifiable information
• multilingual benchmark
• privacy-preserving data sharing
• synthetic data

Innovation

• multilingual anonymization
• synthetic data
• neural machine translation
• personally identifiable information (PII)
• privacy-preserving benchmark