SecureSpeech: Prompt-based Speaker and Content Protection

2025-07-10

AI Summary

To address dual privacy leakage—speaker identity and semantic content—in speech, this paper proposes the first end-to-end prompt-driven speech anonymization framework. Methodologically, it achieves speaker de-identification via controllable speaker descriptors, identifies and replaces sensitive semantic content using named entity recognition and large language models, and synthesizes high-fidelity anonymous speech via text-to-speech. Its key contributions are: (1) the first joint, controllable anonymization of both identity and semantics; (2) a descriptor modulation mechanism that explicitly models the privacy–utility trade-off; and (3) systematic analysis revealing how descriptor selection induces inherent biases between privacy protection and speech usability. Experiments demonstrate strong efficacy: speaker identification accuracy drops below 15%, while ASR accuracy remains above 92% and MOS for naturalness reaches ≈4.1—confirming both effectiveness and practicality.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Speech
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Given the increasing privacy concerns from identity theft and the re-identification of speakers through content in the speech field, this paper proposes a prompt-based speech generation pipeline that ensures dual anonymization of both speaker identity and spoken content. This is addressed through 1) generating a speaker identity unlinkable to the source speaker, controlled by descriptors, and 2) replacing sensitive content within the original text using a name entity recognition model and a large language model. The pipeline utilizes the anonymized speaker identity and text to generate high-fidelity, privacy-friendly speech via a text-to-speech synthesis model. Experimental results demonstrate an achievement of significant privacy protection while maintaining a decent level of content retention and audio quality. This paper also investigates the impact of varying speaker descriptions on the utility and privacy of generated speech to determine potential biases.

Problem

• Protects speaker identity from theft and re-identification
• Anonymizes sensitive content in speech using NLP models
• Balances privacy protection with speech quality and utility

Innovation

• Prompt-based speech generation for anonymization
• Descriptor-controlled speaker identity generation
• NER and LLM for sensitive content replacement