The Cross-Domain Generalization Cost of Offensive Language Detection

2026-07-26

AI Summary

This work addresses the significant performance degradation of offensive language detection models in cross-dataset and cross-lingual settings, where systematic attribution and mitigation mechanisms remain lacking. The authors propose a diagnostic and optimization framework that, for the first time, decomposes cross-domain performance drop into independently measurable dataset and language effects. They introduce zero-shot loss decomposition, a controlled fine-tuning protocol, and a joint training strategy combining temperature-scaled sampling with experience replay. Experiments reveal that dataset effects predominantly drive performance decline; few-shot fine-tuning without replay incurs source-task degradation 4–9 times greater than joint training. The proposed joint approach achieves a Pareto-optimal trade-off, sacrificing only 3.2–4.1 percentage points in source performance to gain 8.1–42.6 percentage points in multilingual capability.

Abstract

Offensive language detection models generally suffer performance degradation when deployed across datasets and across languages, yet most existing studies stop at reporting this phenomenon and lack a systematic methodology for decomposing the causes of degradation into attributable components and quantifying the cost of remediation. This paper proposes a diagnosis and optimization framework composed of three coordinated technical components. First, a zero-shot transfer loss decomposition that separates the performance degradation from OLID to MLMA into two independently measurable components, namely dataset effect and language effect. Second, a controlled fine-tuning protocol that quantifies both adaptation efficiency and the hidden damage inflicted on the source task by comparing few shot learning curves under continued fine-tuning and cold-start starting points. Third, three joint training strategies incorpo rating temperature sampling and experience replay, which offer a controllable Pareto trade-off between improving multilingual capability and preserving source-task performance. Experiments built on this framework show that the dataset effect dominates the zero-shot transfer loss and substantially outweighs the language effect. Few-shot adaptation without a replay mechanism, though data-efficient, inflicts source task damage 4 to 9 times greater than that of the joint training strategies, and its damage magnitude is highly unstable. The three joint training strategies trade 3.2 to 4.1 percentage points of source-task performance for 8.1 to 42.6 percentage points of multilingual capability gain, forming a clear and controllable Pareto trade-off.

Problem

• offensive language detection
• cross-domain generalization
• zero-shot transfer
• multilingual adaptation
• performance degradation

Innovation

• zero-shot transfer loss decomposition
• controlled fine-tuning protocol
• joint training strategies
• experience replay
• Pareto trade-off