A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage

2025-04-28

AI Summary

This paper identifies a pervasive “privacy illusion” in text anonymization: existing methods—such as PII removal or synthetic data generation—only mitigate explicit identifier leakage, yet remain vulnerable to semantic re-identification attacks. To address this, we propose the first risk assessment framework for semantic-level privacy leakage, integrating empirical re-identification attacks, benchmarking against mainstream commercial PII detection tools (e.g., Azure), differential privacy baselines, and evaluation on the MedQA medical QA dataset. Results reveal that Azure fails to protect 74% of sensitive information in MedQA; while differential privacy reduces re-identification risk, it severely degrades textual utility. Our work challenges prevailing assumptions about anonymization efficacy and establishes a verifiable, empirically grounded evaluation paradigm for semantic privacy protection.

Technology Category

Machine Learning: PrivacyNatural Language Processing: Ethics — Bias, Fairness, Transparency & PrivacyComputer Vision: Bias, Fairness & Privacy

Application Category

Security and Privacy: Data transparency and provenanceUser Modeling, Personalization and Recommendation: User privacy protection in personalized systemsSemantics and Knowledge: Provenance, trust, security and privacy, and ethical issues in managing semantic data

Abstract

Sanitizing sensitive text data typically involves removing personally identifiable information (PII) or generating synthetic data under the assumption that these methods adequately protect privacy; however, their effectiveness is often only assessed by measuring the leakage of explicit identifiers but ignoring nuanced textual markers that can lead to re-identification. We challenge the above illusion of privacy by proposing a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release. Our approach shows that seemingly innocuous auxiliary information -- such as routine social activities -- can be used to infer sensitive attributes like age or substance use history from sanitized data. For instance, we demonstrate that Azure's commercial PII removal tool fails to protect 74% of information in the MedQA dataset. Although differential privacy mitigates these risks to some extent, it significantly reduces the utility of the sanitized text for downstream tasks. Our findings indicate that current sanitization techniques offer a extit{false sense of privacy}, highlighting the need for more robust methods that protect against semantic-level information leakage.

Problem

• Evaluating privacy risks beyond explicit identifiers in sanitized text data
• Assessing re-identification attacks using nuanced textual markers for privacy breaches
• Balancing privacy protection and data utility in text sanitization techniques

Innovation

• Proposes framework evaluating re-identification attacks
• Identifies auxiliary information risks in sanitized data
• Highlights need for robust semantic-level protection