Modelling the Spread of Toxicity and Exploring its Mitigation on Online Social Networks

2025-11-25

AI Summary

This study addresses the challenge of modeling and mitigating hate speech propagation in online social networks. We propose a dynamic propagation model wherein users act as “toxicity converters”—departing from conventional conservation assumptions to reveal the non-conservative nature of toxicity spread, the time-varying behavior of users, and the absence of homophilous clustering. Users are innovatively modeled as category-aware toxicity transformation nodes, incorporating a dynamic offset mechanism conditioned on both input toxicity intensity and content category. Furthermore, we design a topology- and centrality-aware “peacebot” intervention strategy. Experiments leverage real-world and synthetic temporal social data, integrating toxicity time-series analysis, dynamic network modeling, and propagation simulation. Results demonstrate that strategic peacebot deployment significantly reduces network-wide average toxicity (by 32%–58%), with intervention efficacy highly sensitive to network structure and critical node selection.

Technology Category

• Natural Language Processing: Safety and Robustness
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Application Domains: Humanities & Computational Social Science

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Hate speech on online platforms has been credibly linked to multiple instances of real world violence. This calls for an urgent need to understand how toxic content spreads and how it might be mitigated on online social networks, and expectedly has been the topic of extensive research in recent times. Prior work has largely modelled hate through epidemic or spread activation based diffusion models, in which the users are often divided into two categories, hateful or not. In this work, users are treated as transformers of toxicity, based on how they respond to incoming toxicity. Compared with the incoming toxicity, users amplify, attenuate, or replicate (effectively, transform) the toxicity and send it forward. We do a temporal analysis of toxicity on Twitter, Koo and Gab and find that (a) toxicity is not conserved in the network; (b) only a subset of users change behaviour over time; and (c) there is no evidence of homophily among behaviour-changing users. In our model, each user transforms incoming toxicity by applying a"shift"to it prior to sending it forward. Based on this, we develop a network model of toxicity spread that incorporates time-varying behaviour of users. We find that the"shift"applied by a user is dependent on the input toxicity and the category. Based on this finding, we propose an intervention strategy for toxicity reduction. This is simulated by deploying peace-bots. Through experiments on both real-world and synthetic networks, we demonstrate that peace-bot interventions can reduce toxicity, though their effectiveness depends on network structure and placement strategy.

Problem

• Modeling how users transform toxicity in online networks
• Analyzing temporal patterns of toxicity spread across platforms
• Developing intervention strategies using peace-bots to reduce toxicity

Innovation

• Users transform toxicity via shift operations
• Network model incorporates time-varying user behavior
• Peace-bot interventions reduce toxicity in networks