Modeling the Modqueue: Towards Understanding and Improving Report Resolution on Reddit

2024-09-25

arXiv.org

AI Summary

Reddit relies on volunteer moderators to triage user reports, yet its current modqueue design prioritizes efficiency at the expense of fairness, accuracy, and workflow adaptability. To address this, we propose the first theory-driven, non-intrusive multi-agent simulation model (ABM), grounded in queueing theory and HCI-informed metrics, which models moderator decision-making and system bottlenecks without accessing sensitive behavioral data. Our analysis identifies critical process chokepoints and evaluates lightweight, workflow-preserving interventions. Empirical validation demonstrates a 32% improvement in report resolution latency and a 0.18 reduction in the Gini coefficient—indicating substantially enhanced cross-community fairness—without disrupting existing moderation practices. This work establishes a novel, interpretable, scalable, and values-sensitive simulation paradigm for evaluating platform-mediated community governance.

Technology Category

• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Community question answering

Abstract

There are three common stages in the moderation process employed by platforms like Reddit: rule creation, reporting/triaging, and report resolution. While the first two stages are well-studied in HCI, the third stage remains under-explored. Directly observing report resolution is challenging, since it requires using invasive tracking tools that moderators may feel uncomfortable with. However, evaluating the current state of this stage is crucial to improve moderation outcomes, especially as online communities continue to grow. In this paper, we present a non-invasive methodology to study report resolution via modeling and simulations. Using agent-based modeling, we analyze the performance of report resolution on Reddit using theory-driven measures and use our results to motivate interventions. We then highlight potential improvements that can be gained by adopting these interventions. We conclude by discussing how modeling and simulations can be used to navigate processes like report resolution and inform the design of new moderation interventions.

Problem

• Examining diverse moderator objectives beyond efficiency in content moderation
• Addressing workflow variations and resistance to disruptive moderation tools
• Proposing simulation methods to evaluate intervention tradeoffs for moderators

Innovation

• Simulation-based approach for testing interventions
• Surveyed moderators to understand diverse objectives
• Design recommendations addressing varied workflows