Native Multilingual Chain-of-Thought Reasoning in Low-Resource Southeast Asian Languages

2026-08-01

AI Summary

This work addresses the issue of cross-lingual collapse in large language models when performing complex reasoning in low-resource Southeast Asian languages, where intermediate reasoning steps often regress to English, undermining localized reasoning capabilities. To mitigate this, the authors propose Onramp-Sequence Cross-Distillation (OSCD), a post-training algorithm that iteratively projects high-resource language reasoning trajectories into low-resource language subspaces via translation agents. OSCD further incorporates a joint embedding mechanism that semantically aligns reasoning trajectories between reference and target languages, enabling stable native chain-of-thought reasoning. The method is the first to prevent cross-lingual collapse in low-resource settings, achieving up to a 3.2× improvement in mathematical reasoning performance on the AIME25 and HMMT25 benchmarks. Additionally, the semantic alignment component yields up to a 6.4% reduction in language bias compared to pure translation baselines.

Abstract

Large Language Models have achieved substantial progress in reasoning capabilities. Yet in low-resource native settings, many suffer from cross-lingual collapse, reverting to English during intermediate steps that require complex logical reasoning. This presents a cold-start bottleneck for policy optimization, whereas standard fine-tuning risks catastrophic forgetting due to cross-lingual representation drift. To address these challenges, we introduce the Onramp-Sequence Cross-Distillation (OSCD), a post-training algorithm that projects high-resource reasoning trajectories into low-resource vocabulary subspaces during generative training rollouts via an integrated translator agentic loop, ensuring the stable and efficient translation of dynamically generated reference samples for fine-tuning. This is coupled with joint-embedding semantic alignment of both reference and target-language reasoning traces, thereby bridging the pairwise cross-lingual representational gaps. Comprehensive evaluations using the AIME25 and HMMT25 benchmarks demonstrate that OSCD yields up to 3.2 times overall improvements in native Southeast Asian languages for mathematical reasoning, of which the joint-embedding semantic alignment component contributes up to 6.4% improvements in linguistic debiasing over translation-only baselines.

Problem

• cross-lingual collapse
• low-resource languages
• reasoning
• catastrophic forgetting
• Southeast Asian languages

Innovation

• cross-lingual reasoning
• low-resource languages
• chain-of-thought
• semantic alignment
• post-training distillation