Inference Attacks for X-Vector Speaker Anonymization

2025-05-13

AI Summary

This work addresses the reliance of privacy–utility trade-off evaluation in x-vector-based voice anonymization on complex speaker recognition models. We propose a lightweight, machine learning–free inference attack that exploits the geometric properties of x-vectors—specifically, pairwise distance analysis and unsupervised clustering—to perform de-anonymization, eliminating the need for deep learning or supervised training. As the first ML-free anonymization assessment paradigm, it achieves significantly higher attack success rates than state-of-the-art model-based attacks across multiple benchmarks. Our results expose a fundamental privacy vulnerability: contemporary x-vector anonymization schemes can be compromised using simple, interpretable statistical methods. This challenges the assumed robustness of such systems and establishes a new evaluation standard that is computationally efficient, fully interpretable, and accessible without specialized ML expertise.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

We revisit the privacy-utility tradeoff of x-vector speaker anonymization. Existing approaches quantify privacy through training complex speaker verification or identification models that are later used as attacks. Instead, we propose a novel inference attack for de-anonymization. Our attack is simple and ML-free yet we show experimentally that it outperforms existing approaches.

Problem

• Study privacy-utility tradeoff in x-vector anonymization
• Propose simple ML-free de-anonymization attack method
• Demonstrate superior performance over existing approaches

Innovation

• Proposes novel inference attack for de-anonymization
• Uses simple and ML-free attack approach
• Outperforms existing complex verification models