Can Emotion Fool Anti-spoofing?

2025-05-29

AI Summary

Current spoofing countermeasure (CM) methods are primarily designed for neutral synthetic speech and exhibit limited robustness against emotionally expressive text-to-speech (TTS) attacks. This work systematically identifies, for the first time, the critical impact of emotional expression on CM performance. We introduce EmoSpoof-TTS—the first benchmark dataset for emotion-aware TTS spoofing detection—and propose the Gated Ensemble Model (GEM), which jointly leverages emotion-specialized submodels and a speech emotion recognition (SER)-guided gating network to enable dynamic, emotion-aware spoofing detection. Our study establishes the novel paradigm of “emotion-aware anti-spoofing.” Evaluated on EmoSpoof-TTS, GEM achieves a 38.2% average reduction in equal error rate (EER) over baseline methods, significantly improving detection accuracy across all emotional categories as well as for neutral speech. The EmoSpoof-TTS dataset and GEM source code are publicly released.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Humans and AI: Emotional Intelligence

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Traditional anti-spoofing focuses on models and datasets built on synthetic speech with mostly neutral state, neglecting diverse emotional variations. As a result, their robustness against high-quality, emotionally expressive synthetic speech is uncertain. We address this by introducing EmoSpoof-TTS, a corpus of emotional text-to-speech samples. Our analysis shows existing anti-spoofing models struggle with emotional synthetic speech, exposing risks of emotion-targeted attacks. Even trained on emotional data, the models underperform due to limited focus on emotional aspect and show performance disparities across emotions. This highlights the need for emotion-focused anti-spoofing paradigm in both dataset and methodology. We propose GEM, a gated ensemble of emotion-specialized models with a speech emotion recognition gating network. GEM performs effectively across all emotions and neutral state, improving defenses against spoofing attacks. We release the EmoSpoof-TTS Dataset: https://emospoof-tts.github.io/Dataset/

Problem

• Traditional anti-spoofing neglects emotional speech variations
• Existing models struggle with emotional synthetic speech attacks
• Emotion-focused anti-spoofing paradigms are urgently needed

Innovation

• EmoSpoof-TTS corpus for emotional synthetic speech
• GEM gated ensemble of emotion-specialized models
• Speech emotion recognition gating network