Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning

2026-06-04

AI Summary

This work addresses the inefficiencies of large language models in multilingual tasks, where blind translation wastes resources and overconfidence impedes effective tool use. The authors propose a rule-free, language-adaptive introspection mechanism that employs reinforcement learning to train a single policy which triggers translation only when comprehension is insufficient, thereby enabling intelligent, domain- and language-aware decisions. Built upon answer-preserving translation data, the approach leverages a confidence-gated GSPO algorithm for continual reinforcement learning on Qwen3-4B across 22 languages and 5 domains. Experiments show the policy achieves up to +23.5 reward improvement on low-resource languages, maintains comparable performance at 63% of the translation cost, and attains Pareto optimality in 87% of cost-sensitive scenarios, significantly outperforming baselines while supporting zero-shot transfer to unseen languages.

Abstract

The performance gap across languages in LLMs is well documented, and closing it natively requires pretraining or fine-tuning on corpora that, for most languages, do not exist. Translation offers an alternative: converting an input into the model's dominant language unlocks its full capabilities at once. Applying translation to every input, however, is wasteful for languages the model already handles, while leaving the choice to the model fails in the opposite way, as LLMs are overconfident and skip the tool even when they cannot understand the input. Prior work resolves this with language-specific rules, domain heuristics, language identifiers, or external routers, each requiring manual engineering. We instead learn a single policy that decides when to translate from reward alone, developing language- and domain-adaptive introspection that assesses its own comprehension and invokes translation only when it cannot solve a task natively. Using data built by our answer-preserving translation pipeline, we continue RL on the post-trained Qwen3-4B across 22 languages in 3 resource tiers (High, Low, XLow) and 5 domains, and introduce confidence-gated GSPO for cost-sensitive tool use. The gated policy lifts reward over the baseline by +4.6 on High, +23.5 on Low, and +17.5 on XLow. Against an unconstrained policy that almost always translates, it preserves full reward at 63% of the cost and is Pareto-optimal across 87% of the cost-sensitivity range. Additionally, to simulate behavior on a completely unseen language, we create 2 synthetic languages, where our gated policy improves +18.7 over the overconfident baseline that underutilizes the tool even on these incomprehensible inputs. The policy transfers zero-shot to 9 held-out languages, and we analyze how tool use emerges over training, per language and per domain.

Problem

• translation tool use
• language performance gap
• cost-aware decision
• overconfidence in LLMs
• multilingual adaptation

Innovation

• cost-aware tool use
• reinforcement learning
• language-adaptive introspection
• confidence-gated GSPO
• zero-shot transfer