ViSpeechFormer: A Phonemic Approach for Vietnamese Automatic Speech Recognition

2026-02-10

AI Summary

This work proposes the first end-to-end automatic speech recognition (ASR) framework with explicit phoneme modeling tailored for Vietnamese, addressing the poor generalization on out-of-vocabulary words and sensitivity to training data bias. Leveraging the highly transparent grapheme-to-phoneme correspondence inherent in Vietnamese orthography, the approach explicitly incorporates phoneme-level representations into a Transformer-based architecture to enhance the model’s capacity to capture pronunciation regularities. Experimental results on two public Vietnamese ASR datasets demonstrate significant performance improvements, particularly in out-of-vocabulary scenarios, where the model exhibits stronger generalization. Moreover, the proposed method effectively mitigates the adverse effects of training data bias, offering a novel paradigm for languages characterized by phonemic orthographies.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Vietnamese has a phonetic orthography, where each grapheme corresponds to at most one phoneme and vice versa. Exploiting this high grapheme-phoneme transparency, we propose ViSpeechFormer (\textbf{Vi}etnamese \textbf{Speech} Trans\textbf{Former}), a phoneme-based approach for Vietnamese Automatic Speech Recognition (ASR). To the best of our knowledge, this is the first Vietnamese ASR framework that explicitly models phonemic representations. Experiments on two publicly available Vietnamese ASR datasets show that ViSpeechFormer achieves strong performance, generalizes better to out-of-vocabulary words, and is less affected by training bias. This phoneme-based paradigm is also promising for other languages with phonetic orthographies. The code will be released upon acceptance of this paper.

Problem

• Vietnamese Automatic Speech Recognition
• phoneme-based ASR
• grapheme-phoneme transparency
• out-of-vocabulary words
• training bias

Innovation

• phoneme-based ASR
• Vietnamese speech recognition
• grapheme-phoneme transparency
• ViSpeechFormer
• out-of-vocabulary generalization