Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models

2026-05-29

AI Summary

This work addresses the challenge that existing confidence estimation methods for large language models are predominantly limited to English and struggle to generalize across languages without retraining. The authors discover that multilingual large language models harbor a cross-lingual shared confidence subspace within their intermediate layers. Leveraging this insight, they propose a lightweight linear probe trained solely on monolingual (e.g., English) supervision to enable zero-shot cross-lingual confidence estimation. Remarkably, the method requires neither target-language data nor model retraining, yet achieves strong generalization across diverse unseen languages. It significantly outperforms current state-of-the-art confidence estimation approaches and represents the first successful demonstration of cross-lingual confidence transfer without any supervision in the target languages.

Abstract

Confidence estimation (CE), i.e. quantifying the reliability of a model's prediction, has attracted great interest in the context of large language models (LLMs). However, most studies focus on English, ignoring the multilingual reality of LLM usage, while many CE methods degrade or require retraining across languages. To address this gap, we investigate whether multilingual LLMs encode shared, language-transferable confidence features. We use a lightweight linear probe that predicts answer correctness directly from intermediate representations. Trained monolingually, the probe generalizes zero-shot to unseen, typologically diverse languages without target-language supervision. Learned layer weights and multiple ablations reveal that confidence features concentrate in middle layers across languages, suggesting a shared confidence subspace. While zero-shot cross-lingual performance depends on similarity to the source language, the probe provides a strong baseline without any retraining and compares favorably to other popular confidence estimation methods.

Problem

• confidence estimation
• cross-lingual
• zero-shot
• multilingual language models
• language transfer

Innovation

• zero-shot cross-lingual
• confidence estimation
• linear probe
• shared confidence subspace
• multilingual LLMs