Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models

2025-08-15

AI Summary

Arabic social media content poses significant challenges for detecting hope, hate speech, offensive language, and fine-grained sentiment due to linguistic complexity and insufficient multimodal analysis—particularly for image-text memes. Method: This work proposes a large language model (LLM)-based multimodal joint modeling framework that integrates GPT-4o-mini and Gemini Flash 2.5, enhanced by instruction tuning, cross-modal alignment embeddings, and domain adaptation strategies to enable synergistic text-image semantic understanding. Contribution/Results: We introduce the first Arabic fine-grained attitude–sentiment joint classification framework. At the ArabicNLP MAHED 2025 Challenge, our approach achieved first place across all three core tasks—hope detection, hate speech identification, and sentiment classification—with macro-F1 scores of 72.1%, 57.8%, and 79.6%, respectively—substantially advancing the automation and precision of Arabic content safety governance.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The rise of social media and online communication platforms has led to the spread of Arabic textual posts and memes as a key form of digital expression. While these contents can be humorous and informative, they are also increasingly being used to spread offensive language and hate speech. Consequently, there is a growing demand for precise analysis of content in Arabic text and memes. This paper explores the potential of large language models to effectively identify hope, hate speech, offensive language, and emotional expressions within such content. We evaluate the performance of base LLMs, fine-tuned LLMs, and pre-trained embedding models. The evaluation is conducted using a dataset of Arabic textual speech and memes proposed in the ArabicNLP MAHED 2025 challenge. The results underscore the capacity of LLMs such as GPT-4o-mini, fine-tuned with Arabic textual speech, and Gemini Flash 2.5, fine-tuned with Arabic memes, to deliver the superior performance. They achieve up to 72.1%, 57.8%, and 79.6% macro F1 scores for tasks 1, 2, and 3, respectively, and secure first place overall in the Mahed 2025 challenge. The proposed solutions offer a more nuanced understanding of both text and memes for accurate and efficient Arabic content moderation systems.

Problem

• Detecting hope, hate, and emotion in Arabic text and memes
• Analyzing offensive language and hate speech in digital content
• Developing accurate Arabic content moderation systems using LLMs

Innovation

• Fine-tuned LLMs for Arabic text analysis
• Multi-modal meme processing with Gemini Flash
• Superior performance in Arabic content moderation