FlashTTS: Fast Streaming TTS with MTP Acceleration and X-pred Mean Flow Distillation

2026-06-08

AI Summary

Existing text-to-speech (TTS) systems struggle to simultaneously achieve low latency and streaming capability due to autoregressive generation or multi-step flow matching. This work proposes FlashTTS, a natively streaming TTS framework that introduces a novel lagging multi-track architecture to eliminate sentence-level buffering. By integrating multi-token parallel prediction (MTP) with an X-pred mean-based flow-matching decoder, FlashTTS enables high-quality non-autoregressive acoustic generation in just two function evaluations. The method reduces first-packet latency to 325 ms—significantly outperforming strong baselines—while preserving excellent zero-shot voice cloning performance and cross-lingual intelligibility.

Abstract

Recent progress in speech dialogue systems requires Text-to-Speech (TTS) models to be faster and more responsive. Modern speech dialogue systems impose two primary requirements on TTS models: low latency and support for streaming inputs and outputs. However, most existing single-codebook LLM-based TTS methods rely on multi-stage pipelines that lack native streaming capabilities. These systems typically suffer from high end-to-end latency due to slow autoregressive prediction and multi-step flow matching. To address these limitations, we propose FlashTTS, an open-source and low-latency streaming TTS framework. FlashTTS introduces a lagged multi-track architecture that natively processes streaming text and speech inputs, thereby eliminating the need for sentence-level buffering. To accelerate acoustic generation, we integrate parallel Multi-Token Prediction (MTP) with an X-pred mean flow matching decoder. This configuration achieves high-fidelity token-to-mel generation in exactly two function evaluations (2-NFE). By jointly optimizing input processing and decoding efficiency, FlashTTS offers a practical foundation for real-time speech dialogue systems. Experiments show that FlashTTS substantially reduces First-Packet Latency to 325ms compared to robust streaming baselines, all while preserving strong zero-shot voice cloning and cross-lingual intelligibility. Speech samples are available. The model code and checkpoints will be released as open source.

Problem

• Text-to-Speech
• streaming
• low latency
• autoregressive prediction
• flow matching

Innovation

• Streaming TTS
• Multi-Token Prediction
• Flow Matching
• Low Latency
• 2-NFE Generation