Defining, Understanding, and Detecting Online Toxicity: Challenges and Machine Learning Approaches

2025-09-13

AI Summary

During sensitive periods—such as crises and elections—the detection of online harmful content (e.g., hate speech, offensive language) faces core challenges including conceptual ambiguity and poor generalizability across contexts and languages. Method: This study systematically reviews 140 relevant works to clarify definitional boundaries and data limitations; proposes a novel multilingual, cross-platform toxicity detection paradigm; and introduces a comprehensive benchmark dataset covering 32 languages and high-stakes scenarios—including elections and public health emergencies. Leveraging advanced machine learning and NLP techniques, the study optimizes classification models for enhanced cross-lingual and cross-platform robustness. Contribution/Results: The framework significantly improves accuracy and generalizability in toxic content identification, offering a reusable methodological foundation and empirically grounded guidelines for real-world content moderation practices.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Multimodal Learning
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews

Abstract

Online toxic content has grown into a pervasive phenomenon, intensifying during times of crisis, elections, and social unrest. A significant amount of research has been focused on detecting or analyzing toxic content using machine-learning approaches. The proliferation of toxic content across digital platforms has spurred extensive research into automated detection mechanisms, primarily driven by advances in machine learning and natural language processing. Overall, the present study represents the synthesis of 140 publications on different types of toxic content on digital platforms. We present a comprehensive overview of the datasets used in previous studies focusing on definitions, data sources, challenges, and machine learning approaches employed in detecting online toxicity, such as hate speech, offensive language, and harmful discourse. The dataset encompasses content in 32 languages, covering topics such as elections, spontaneous events, and crises. We examine the possibility of using existing cross-platform data to improve the performance of classification models. We present the recommendations and guidelines for new research on online toxic consent and the use of content moderation for mitigation. Finally, we present some practical guidelines to mitigate toxic content from online platforms.

Problem

• Defining and detecting online toxic content
• Challenges in automated toxicity detection mechanisms
• Improving classification models with cross-platform data

Innovation

• Machine learning for toxic content detection
• Cross-platform data integration for classification
• Multilingual dataset analysis for toxicity