Efficient Toxicity Detection in Gaming Chats: A Comparative Study of Embeddings, Fine-Tuned Transformers and LLMs

2025-10-20

AI Summary

This study addresses the challenge of real-time, low-cost detection of toxic content in online gaming chat. We systematically evaluate four NLP paradigms—word embeddings, zero-shot/few-shot large language model (LLM) inference, fine-tuned Transformer models (including DistilBERT), and retrieval-augmented generation (RAG)—along three dimensions: classification accuracy, inference latency, and computational cost. We propose a hybrid moderation architecture integrating automated detection with continual learning to substantially reduce human review burden. Experimental results show that fine-tuned DistilBERT achieves 92.3% accuracy while improving inference speed by 4.8× and reducing GPU memory consumption by 67%, outperforming all alternatives. Its lightweight design and robustness make it the optimal choice for dynamic gaming environments. To our knowledge, this is the first end-to-end, multi-paradigm NLP benchmark conducted specifically in the gaming context, empirically identifying the Pareto-optimal trade-off between accuracy and deployment efficiency for lightweight fine-tuned models.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper presents a comprehensive comparative analysis of Natural Language Processing (NLP) methods for automated toxicity detection in online gaming chats. Traditional machine learning models with embeddings, large language models (LLMs) with zero-shot and few-shot prompting, fine-tuned transformer models, and retrieval-augmented generation (RAG) approaches are evaluated. The evaluation framework assesses three critical dimensions: classification accuracy, processing speed, and computational costs. A hybrid moderation system architecture is proposed that optimizes human moderator workload through automated detection and incorporates continuous learning mechanisms. The experimental results demonstrate significant performance variations across methods, with fine-tuned DistilBERT achieving optimal accuracy-cost trade-offs. The findings provide empirical evidence for deploying cost-effective, efficient content moderation systems in dynamic online gaming environments.

Problem

• Comparing NLP methods for toxicity detection in gaming chats
• Evaluating accuracy, speed and computational cost trade-offs
• Developing efficient hybrid moderation systems for gaming environments

Innovation

• Fine-tuned DistilBERT optimizes accuracy-cost trade-offs
• Hybrid system automates detection and reduces moderator workload
• Comparative evaluation of embeddings, transformers, and LLMs