Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment $\unicode{x2013}$ Is English Enough?

2026-08-04

AI Summary

This study investigates whether English alone is sufficient as a cross-lingual alignment benchmark for predicting the performance of large language models (LLMs) on non-English classification and translation tasks. The authors systematically evaluate 27 variants of cross-lingual alignment (CLA) scores and, for the first time, apply them to machine translation performance prediction. They also introduce a language-agnostic PMI-based translation evaluation metric that reduces reliance on the target language. Experimental results reveal that alignment based solely on English performs comparably to, or even better than, source–target language alignment, supporting the hypothesis that LLMs internally use English as a pivot language. Furthermore, the proposed PMI-based metric shows high correlation with chrF, offering a novel tool and empirical foundation for evaluating multilingual models.

Abstract

Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given language are more aligned to English within the model. Several cross-lingual alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target languages, we propose a PMI-based translation metric, which is less dependent on the target language and correlates strongly with chrF. We find that CLA with English predicts translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot language.

Problem

• cross-lingual alignment
• multilingual LLMs
• translation performance
• classification performance
• English pivot

Innovation

• cross-lingual alignment
• multilingual LLMs
• translation evaluation
• PMI-based metric
• pivot language