Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study

2026-08-04

AI Summary

This work addresses the high memory and computational costs of multilingual neural machine translation models stemming from their large vocabularies, a challenge inadequately tackled by existing compression methods that fail to effectively eliminate lexical redundancy. The authors propose a corpus-driven framework that synergistically combines vocabulary pruning with targeted fine-tuning, thereby breaking free from reliance on the original vocabulary structure for the first time. Applied to prominent models such as M2M100, NLLB-200, and mBART-50, the approach reduces vocabulary size from 128,000 to approximately 10,000 tokens—cutting memory usage by 60%—while simultaneously enhancing translation quality. Notably, the pruned M2M100 model surpasses specialized bilingual baselines, achieving state-of-the-art performance with BLEU and COMET scores of 42.04 and 0.8730, respectively, thus demonstrating both efficiency and high fidelity in multilingual translation.

Abstract

The adoption of large pre-trained multilingual models for neural machine translation (MNMT) faces a major challenge: excessive memory and computational consumption due to overly large vocabularies and embedding layers. Although existing compression methods like pruning, quantization and knowledge distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance. Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 (against 44.59 for the OPUS-MTen- ar bilingual model) while it significantly outperforms it on the COMET metric (0.8730 vs 0.7911) revealing superior semantic adequacy and fluency.

Problem

• multilingual neural machine translation
• vocabulary size
• memory efficiency
• computational cost
• embedding layers

Innovation

• vocabulary pruning
• multilingual neural machine translation
• corpus-driven optimization
• efficient NMT
• targeted fine-tuning