MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition

2026-07-27

AI Summary

This work addresses the "curse of multilinguality" in large-scale speech recognition, where model capacity is diluted across many languages, degrading performance for each. The authors propose a language-family-based mixture-of-experts mechanism built upon a self-supervised speech model (S3M). By leveraging interpretable language clustering to group linguistically similar languages, the approach assigns dedicated expert modules to each cluster. Additionally, hierarchical low-rank adaptation (LoRA) is introduced within an acoustic–linguistic disentangled architecture to efficiently capture language-specific characteristics. The method substantially reduces the number of required experts and achieves significant gains over dense multilingual baselines on a benchmark covering 495 languages, with only a marginal increase in trainable parameters while simultaneously improving both phonemic and orthographic modeling.

Abstract

Massively multilingual automatic speech recognition (ASR) models covering hundreds of languages must maintain robust performance across diverse linguistic and acoustic conditions. However, these models often encounter the curse of multilinguality, where model capacity is diluted across languages. To address this challenge, we propose Mixture of Language Group Experts (MoLGE), built upon speech self-supervised models (S3Ms). MoLGE assigns dedicated expert modules to clusters of similar languages, reducing the number of required submodules compared to conventional language-specific Mixture-of-Experts (MoE) schemes. It further integrates a hierarchical Low-Rank Adaptation (LoRA) strategy into the disentangled acoustic and linguistic components of the S3M architecture, enabling efficient modeling of language-specific characteristics while maintaining parameter efficiency. Further, we investigate the impact of language grouping strategies based on both linguistic and data-driven criteria on overall performance, providing an interpretable perspective on how language structure influences scalability in multilingual speech systems. In experiments, we evaluate MoLGE on a multilingual benchmark encompassing 495 languages. Results demonstrate that MoLGE consistently outperforms dense multilingual baselines with a minimal increase in trainable parameters. Notably, these language grouping strategies yield substantial improvements for both phonetic and orthographic aspects of ASR modeling. Our findings suggest that structured language specialization provides an effective pathway for massively scaling language coverage of multilingual ASR.

Problem

• massively multilingual ASR
• curse of multilinguality
• language-specific modeling
• scalability
• speech recognition

Innovation

• Mixture of Experts
• Language Grouping
• Low-Rank Adaptation
• Multilingual ASR
• Self-Supervised Speech Models