Technology Category
Natural Language Processing: SpeechHumans and AI: Emotional IntelligenceMachine Learning: Large Multimodal Models (LMMs)
EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vector
🤖 AI Summary
Existing emotional text-to-speech (TTS) systems face challenges including high emotional expressiveness complexity, scarcity of annotated emotional speech data, and poor cross-speaker generalization. This paper proposes a zero-shot emotion-controllable TTS framework that synthesizes natural speech with diverse emotion types and intensities across speakers—without requiring any emotional speech data or manual annotations from the target speaker. Our key contributions are: (1) the first emotion-adaptive spherical vector representation, unifying emotion style and intensity in a geometrically principled manner; (2) a hierarchical style encoder coupled with an enhanced loss function explicitly designed for zero-shot style transfer; and (3) a conditional flow-matching decoder to improve generation fidelity and prosodic naturalness. Extensive experiments on multiple benchmarks demonstrate significant improvements in both emotion accuracy and speech naturalness, achieving high-quality emotional speech synthesis with only a few sampling steps.
Technology Category
Application Category
📝 Abstract
Emotional text-to-speech (TTS) technology has achieved significant progress in recent years; however, challenges remain owing to the inherent complexity of emotions and limitations of the available emotional speech datasets and models. Previous studies typically relied on limited emotional speech datasets or required extensive manual annotations, restricting their ability to generalize across different speakers and emotional styles. In this paper, we present EmoSphere++, an emotion-controllable zero-shot TTS model that can control emotional style and intensity to resemble natural human speech. We introduce a novel emotion-adaptive spherical vector that models emotional style and intensity without human annotation. Moreover, we propose a multi-level style encoder that can ensure effective generalization for both seen and unseen speakers. We also introduce additional loss functions to enhance the emotion transfer performance for zero-shot scenarios. We employ a conditional flow matching-based decoder to achieve high-quality and expressive emotional TTS in a few sampling steps. Experimental results demonstrate the effectiveness of the proposed framework.
Problem
Research questions and friction points this paper is trying to address.
Control emotional style and intensity in zero-shot TTS
Model emotions without human annotation requirements
Generalize across seen and unseen speaker styles
Innovation
Methods, ideas, or system contributions that make the work stand out.
Emotion-adaptive spherical vector for annotation-free modeling
Multi-level style encoder for speaker generalization
Conditional flow matching decoder for efficient sampling
🔎 Similar Papers
No similar papers found.
