Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey

2026-05-13

AI Summary

Existing monolingual knowledge editing methods suffer from cross-lingual interference and degraded performance in multilingual settings. This study systematically evaluates six vector fusion strategies for large-scale batch editing, conducting experiments on the MzsRE benchmark across two prominent large language models, two foundational editing approaches, and twelve languages. The findings reveal that vector addition with shared covariance emerges as the most robust fusion strategy, significantly outperforming naive addition without covariance sharing. Temporal-Spatial Vector Mapping (TSVM) demonstrates limited efficacy in mitigating multilingual interference. Moreover, moderately increasing the weight scaling factor while employing a lower-rank update substantially enhances editing performance. This work is the first to uncover the critical roles of weight scaling and rank compression in multilingual knowledge editing, offering clear practical guidance for developing effective multilingual editing systems.

Abstract

Multilingual knowledge editing (MKE) remains challenging because language-specific edits interfere with one another, even when locate-then-edit methods work well in monolingual settings. This paper focuses on three issues: the effectiveness of vector merging methods for MKE, the extent to which Task Singular Vectors for Merging (TSVM) can reduce multilingual interference, and the influence of the weight scaling factor and rank compression ratio on performance. We evaluate six merging variants with two popular backbone large language models, two base knowledge editing methods, and 12 languages on the MzsRE benchmark under a large-scale batch-editing setting. Our results show that vector summation with shared covariance is the most reliable overall strategy, whereas simple summation without shared covariance performs poorly. TSVM improves performance in some settings, but its ability to mitigate multilingual interference is limited. We also find that performance is sensitive to both weight scale and rank ratio, with larger-than-default scaling and relatively low rank often yielding better results. These findings clarify the practical strengths and limits of current vector merging methods for MKE and provide guidance for future multilingual knowledge editing research.

Problem

• multilingual knowledge editing
• language interference
• vector merging
• large language models
• knowledge editing

Innovation

• multilingual knowledge editing
• vector merging
• TSVM
• weight scaling
• rank compression