CODEOFCONDUCT at Multilingual Counterspeech Generation: A Context-Aware Model for Robust Counterspeech Generation in Low-Resource Languages

2025-01-01

AI Summary

To address the challenge of generating counter-hate speech for low-resource languages, this paper proposes a context-aware, robust multilingual counter-speech generation model. Methodologically, it innovatively integrates simulated annealing into the multilingual fine-tuning process to jointly optimize cross-lingual generalization and factual consistency. The model builds upon a multilingual pretraining framework and employs an end-to-end optimization strategy guided by a hybrid evaluation metric comprising BLEU, ROUGE, BERTScore, Novelty, and JudgeLM. Evaluated on the MCG-COLING-2025 shared task, our approach achieves state-of-the-art (SOTA) performance across four languages—Basque, English, Italian, and Spanish. Notably, it secures the top three positions in Basque-specific evaluation and ranks first overall, demonstrating superior effectiveness—particularly for low-resource settings—while maintaining linguistic fidelity and contextual relevance.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

This paper introduces a context-aware model for robust counterspeech generation, which achieved significant success in the MCG-COLING-2025 shared task. Our approach particularly excelled in low-resource language settings. By leveraging a simulated annealing algorithm fine-tuned on multilingual datasets, the model generates factually accurate responses to hate speech. We demonstrate state-of-the-art performance across four languages (Basque, English, Italian, and Spanish), with our system ranking first for Basque, second for Italian, and third for both English and Spanish. Notably, our model swept all three top positions for Basque, highlighting its effectiveness in low-resource scenarios. Evaluation of the shared task employs both traditional metrics (BLEU, ROUGE, BERTScore, Novelty) and JudgeLM based on LLM. We present a detailed analysis of our results, including an empirical evaluation of the model performance and comprehensive score distributions across evaluation metrics. This work contributes to the growing body of research on multilingual counterspeech generation, offering insights into developing robust models that can adapt to diverse linguistic and cultural contexts in the fight against online hate speech.

Problem

• Multilingual Environment
• Counter-speech Generation
• Resource-poor Languages

Innovation

• Multilingual Counter-speech Generation
• Annealing-inspired Algorithm
• Cross-lingual Performance Evaluation