Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selection

2026-07-31

AI Summary

This study addresses the underperformance of large language models (LLMs) on Arabic medical tasks compared to English, attributing it not to insufficient training data but to ineffective cross-lingual knowledge transfer. The work introduces the first application of causal mechanism analysis to targeted adaptation of LLMs for low-resource medical languages. By employing tuned lens probing and causal activation patching, the authors identify latent Arabic medical knowledge already present in intermediate layers and apply Targeted Low-Rank Adaptation (TLoRA) within critical representation bifurcation windows for efficient parameter updates. Additionally, they construct AraClinicDialog, the first clinical dialogue benchmark encompassing Modern Standard Arabic and four major dialects. Experiments demonstrate consistent superiority over full-network LoRA, zero-shot, and few-shot baselines across multiple-choice, short-answer generation, and multi-turn dialogue tasks—without requiring task-specific fine-tuning.

Abstract

Large Language Models (LLMs) perform strongly in English medical tasks but degrade substantially in Arabic, a gap widely attributed to limited training data. We systematically investigate this assumption via tuned lens probing and causal activation patching, and find that Arabic medical knowledge is present in intermediate model representations but fails to surface at the output. This mechanistic insight motivates a targeted adaptation strategy: rather than fine-tuning the full network, we propose Targeted Low-Rank Adaptation (TLoRA), restricted to the layer window where cross-lingual representations diverge, upstream of the output layers where the failure manifests. We evaluate TLoRA on multiple-choice medical QA, where our approach outperforms full-network LoRA, zero-shot, and few-shot baselines. We further evaluate it on short-answer generation and multi-turn clinical dialogue, where it performs competitively without the need for task-specific finetuning. We additionally introduce AraClinicDialog, a clinician-constructed Arabic medical dialogue benchmark in MSA with validated variants across four Arabic dialects. Together, these contributions demonstrate that mechanistic diagnosis can serve as a practical guide for targeted adaptation in underrepresented-language medical LLMs.

Problem

• English-Arabic medical knowledge gap
• Large Language Models
• Arabic medical tasks
• underrepresented-language LLMs

Innovation

• Targeted Low-Rank Adaptation
• Causal Layer Selection
• Cross-lingual Medical LLMs
• Mechanistic Interpretability
• AraClinicDialog