Strategic Filtering for Content Moderation: Free Speech or Free of Distortion?

2025-07-26

AI Summary

Automated content moderation on social media often triggers strategic evasion by users, creating a dilemma between restricting free expression and exacerbating societal distortion. Method: This paper is the first to model the platform–user interaction as a mechanism design problem, formally characterizing the trade-off between expressive freedom and societal distortion; it proves that computing the optimal moderation policy is NP-hard. To address this, we propose a computationally tractable framework combining approximation algorithms with offline data analysis, enabling near-optimal policy estimation from limited historical data and providing generalization guarantees. Results: Experiments demonstrate that our approach significantly reduces content manipulation while improving user expressive freedom. It establishes a theoretically grounded, practically implementable foundation for building moderation systems that are both low-distortion and highly inclusive.

Technology Category

• Multiagent Systems: Mechanism Design
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

User-generated content (UGC) on social media platforms is vulnerable to incitements and manipulations, necessitating effective regulations. To address these challenges, those platforms often deploy automated content moderators tasked with evaluating the harmfulness of UGC and filtering out content that violates established guidelines. However, such moderation inevitably gives rise to strategic responses from users, who strive to express themselves within the confines of guidelines. Such phenomena call for a careful balance between: 1. ensuring freedom of speech -- by minimizing the restriction of expression; and 2. reducing social distortion -- measured by the total amount of content manipulation. We tackle the problem of optimizing this balance through the lens of mechanism design, aiming at optimizing the trade-off between minimizing social distortion and maximizing free speech. Although determining the optimal trade-off is NP-hard, we propose practical methods to approximate the optimal solution. Additionally, we provide generalization guarantees determining the amount of finite offline data required to approximate the optimal moderator effectively.

Problem

• Balancing free speech and reducing content manipulation in moderation
• Optimizing trade-off between social distortion and expression freedom
• Approximating optimal moderation solutions with finite offline data

Innovation

• Automated content moderation for UGC regulation
• Mechanism design optimizes free speech vs distortion
• Practical approximation methods for NP-hard problem