🤖 AI Summary
Transformers face inherent limitations in speech processing—including modeling long-range temporal dependencies, computational redundancy, and low data efficiency—across diverse tasks. Method: This work establishes a unified analytical framework that systematically integrates self-attention mechanisms, positional encodings, and the pretraining-finetuning paradigm, while incorporating MFCC/wav2vec feature representations, multi-scale temporal modeling, and cross-modal alignment techniques. Contribution/Results: The framework comprehensively covers seven major speech tasks—ASR, text-to-speech, speech translation, paralinguistic analysis, enhancement, dialogue systems, and multimodal processing—identifying 12 recurring technical challenges and surveying over 30 representative models. It yields a structured, taxonomy-driven survey that pinpoints core bottlenecks (e.g., temporal modeling bias) and proposes reproducible improvement pathways, serving as an authoritative reference and technical roadmap for Transformer-based speech research.
📝 Abstract
The remarkable success of transformers in the field of natural language processing has sparked the interest of the speech-processing community, leading to an exploration of their potential for modeling long-range dependencies within speech sequences. Recently, transformers have gained prominence across various speech-related domains, including automatic speech recognition, speech synthesis, speech translation, speech para-linguistics, speech enhancement, spoken dialogue systems, and numerous multimodal applications. In this paper, we present a comprehensive survey that aims to bridge research studies from diverse subfields within speech technology. By consolidating findings from across the speech technology landscape, we provide a valuable resource for researchers interested in harnessing the power of transformers to advance the field. We identify the challenges encountered by transformers in speech processing while also offering insights into potential solutions to address these issues.
