From SALAMANDRA to SALAMANDRATA: BSC Submission for WMT25 General Machine Translation Shared Task

2025-08-18

AI Summary

This work addresses the limited generalization capability of multilingual machine translation (MT) across both European and non-European languages. To this end, we propose the SALAMANDRATA series of models, which integrate a novel two-stage continual pretraining strategy with high-quality multilingual instruction fine-tuning. We expand the vocabulary to support 38 European languages and newly introduced non-European languages from WMT25, and enhance inference with minimum Bayes risk decoding and COMET/COMET-KIWI–guided re-ranking. Our contributions include: (1) releasing open-weight models with 2B and 7B parameters, along with an improved SALAMANDRATA-V2; (2) achieving significant performance gains on diverse-directional translation tasks in the WMT25 benchmark; and (3) publicly releasing all models on Hugging Face. This work advances the practical deployment of large-scale, robust multilingual MT systems.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this paper, we present the SALAMANDRATA family of models, an improved iteration of SALAMANDRA LLMs (Gonzalez-Agirre et al., 2025) specifically trained to achieve strong performance in translation-related tasks for 38 European languages. SALAMANDRATA comes in two scales: 2B and 7B parameters. For both versions, we applied the same training recipe with a first step of continual pre-training on parallel data, and a second step of supervised fine-tuning on high-quality instructions. The BSC submission to the WMT25 General Machine Translation shared task is based on the 7B variant of SALAMANDRATA. We first adapted the model vocabulary to support the additional non-European languages included in the task. This was followed by a second phase of continual pre-training and supervised fine-tuning, carefully designed to optimize performance across all translation directions for this year's shared task. For decoding, we employed two quality-aware strategies: Minimum Bayes Risk Decoding and Tuned Re-ranking using COMET and COMET-KIWI respectively. We publicly release both the 2B and 7B versions of SALAMANDRATA, along with the newer SALAMANDRATA-V2 model, on Hugging Face1

Problem

• Improving machine translation for 38 European languages
• Adapting model vocabulary for non-European languages
• Optimizing performance with quality-aware decoding strategies

Innovation

• Continual pre-training on parallel data
• Supervised fine-tuning on high-quality instructions
• Quality-aware decoding with MBR and COMET