Cross-Lingual Activation Steering for Multilingual Language Models

2026-01-23

AI Summary

This work addresses the significant performance gap between high-resource and low-resource languages in multilingual large language models, which stems from an imbalance between shared and language-specific neuronal representations. The authors propose a training-free, inference-stage activation modulation method that selectively adjusts neuron activations to dynamically enhance separation among language clusters without altering model weights. Their approach demonstrates that effective cross-lingual transfer arises from functional differentiation rather than strict representational alignment, thereby unlocking the model’s latent multilingual capacity. Experimental results show consistent improvements across diverse tasks, with average gains of 2.3% in accuracy on classification and 3.4% in F1 score on generation tasks, while preserving performance on high-resource languages.

Technology Category

Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLPMachine Learning: Large Multimodal Models (LMMs)Computer Vision: Language and Vision

Application Category

Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web searchWeb Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web dataUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant languages. Prior work attributes this gap to imbalances between shared and language-specific neurons in multilingual representations. We propose Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that selectively modulates neuron activations. We evaluate CLAS on classification and generation benchmarks, achieving average improvements of 2.3% (Acc.) and 3.4% (F1) respectively, while maintaining high-resource language performance. We discover that effective transfer operates through functional divergence rather than strict alignment; performance gains correlate with increased language cluster separation. Our results demonstrate that targeted activation steering can unlock latent multilingual capacity in existing models without modification to model weights.

Problem

• multilingual language models
• performance gap
• cross-lingual transfer
• language imbalance
• neuron activation

Innovation

• Cross-Lingual Activation Steering
• multilingual language models
• neuron activation modulation
• training-free intervention
• language cluster separation