Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection

2026-06-02

AI Summary

This study addresses polarization in multilingual texts by proposing a prompt engineering–based approach for fine-grained detection, encompassing three subtasks: binary polarization classification, polarization type identification, and manifestation categorization. The authors systematically design twelve prompt templates, varying along dimensions such as term clarity, definition specificity, reasoning guidance, and contextual exemplars, thereby offering the first comprehensive evaluation of how prompt design influences multilingual polarization detection. Experiments are conducted on the Aya-101 and Gemma3-27B models, with the latter achieving average macro F1 scores of 0.762, 0.587, and 0.444 across 22 languages. These results delineate both the capabilities and limitations of prompt engineering in coarse- and fine-grained sociolinguistic classification tasks.

Abstract

Our submission presented in this paper is for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection and it covers all three subtasks: (1) binary polarization detection, (2) polarization type classification and (3) polarization manifestation identification. We adopt a systematic approach of research on short designed prompts by considering twelve designed prompts that are different in terminology clarity, detail of the definition, guidance of reasoning and in-context examples use. The experiments are conducted using aya-101 and Gemma3-27B, with the latter chosen for the submission at the end of the development through performance considerations. Our system has an average macro level F1-score of 0.762 on Subtask 1, 0.587 on Subtask 2 and 0.444 on Subtask 3 with the average accuracy of 0.819, 0.678 and 0.498, respectively, on the official test set averaged among 22 languages, respectively. With cross-task and cross-lingual analysis, we demonstrate that prompt-based approaches can be used effectively to detect coarse grained polarization but encounter more and more difficulties as far as fine-grained and multi-label sociolinguistic classification is concerned.

Problem

• polarization detection
• multilingual text classification
• prompt-based approach
• sociolinguistic classification
• SemEval-2026 Task 9

Innovation

• prompt engineering
• polarization detection
• multilingual text classification
• large language models
• sociolinguistic analysis