Problem
Research questions and friction points this paper is trying to address.
Dynamic speaking style adaptation
Context-aware TTS
Instruction-following
Timbre drift
Interactive TTS: Dynamic Speaking Style Adaptation for Expressive Speech Synthesis
🤖 AI Summary
该研究解决了多轮多模态交互中动态说话风格适应的问题，通过显式建模上下文风格决策并引入Iterative RSFT和CADPO方法来提高文本转语音系统的表达力和一致性。
📝 Abstract
Dynamic speaking style adaptation in multi-turn multimodal interaction remains a major challenge for text-to-speech (TTS) systems. Existing context-aware TTS (CTTS) methods typically map dialogue context to speech in an end-to-end manner. Such implicit modeling makes contextual style decisions difficult to supervise, while the entanglement of style, timbre, and content often leads to weak instruction-following and severe timbre drift across turns. To overcome these limitations, we propose Interactive TTS, a dynamic, style-adaptive framework for contextually appropriate and speaker-consistent speech generation. Interactive TTS decouples the process by explicitly modeling contextual style decisions as executable instructions. To bridge the gap between style decisions and speech generation, we introduce Iterative Rejection Sampling Fine-Tuning (Iterative RSFT) and Context-Aware Direct Preference Optimization (CADPO), which significantly enhance instruction-following and align the generated speech with conversational contexts. Extensive experiments demonstrate that Interactive TTS outperforms state-of-the-art models on VStyle and SpeechParaling-Bench. Demo is available at https://wjtian-wonderful.github.io/InteractiveTTS/
Innovation
Methods, ideas, or system contributions that make the work stand out.
Dynamic Speaking Style Adaptation
Executable Instructions for Contextual Styles
Iterative Rejection Sampling Fine-Tuning
Context-Aware Direct Preference Optimization
🔎 Similar Papers
No similar papers found.
