A survey of textual cyber abuse detection using cutting-edge language models and large language models

2025-01-09

AI Summary

Online textual abuse—including hate speech and cyberbullying—seriously harms users’ mental health and erodes social trust. While large language models (LLMs) enhance detection capabilities, they may also generate harmful content, exacerbating governance challenges. This study systematically reviews text abuse detection methods in Chinese social media and introduces, for the first time, a “technical–ethical” co-analysis framework. We empirically evaluate leading LLMs across four critical dimensions: detection accuracy, bias, robustness, and risk of generating abusive content. By integrating text classification, psychosocial impact modeling, and adversarial generation analysis, we uncover the dialectical role of LLMs—both mitigating and amplifying online abuse. Our findings provide empirically grounded, actionable insights for safe AI governance, including a phased technical roadmap for responsible deployment and mitigation.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews

Abstract

The success of social media platforms has facilitated the emergence of various forms of online abuse within digital communities. This abuse manifests in multiple ways, including hate speech, cyberbullying, emotional abuse, grooming, and sexting. In this paper, we present a comprehensive analysis of the different forms of abuse prevalent in social media, with a particular focus on how emerging technologies, such as Language Models (LMs) and Large Language Models (LLMs), are reshaping both the detection and generation of abusive content within these networks. We delve into the mechanisms through which social media abuse is perpetuated, exploring the psychological and social impact. Additionally, we examine the dual role of advanced language models-highlighting their potential to enhance automated detection systems for abusive behavior while also acknowledging their capacity to generate harmful content. This paper aims to contribute to the ongoing discourse on online safety and ethics, offering insights into the evolving landscape of cyberabuse and the technological innovations that both mitigate and exacerbate it.

Problem

• Online Misconduct
• Social Media
• Language Models

Innovation

• AI Language Models
• Cyberbullying Detection
• Psychosocial Impact