Leveraging a Multi-Agent LLM-Based System to Educate Teachers in Hate Incidents Management

2025-06-30

AI Summary

This study addresses educators’ insufficient capacity to identify and intervene in hate incidents within school settings. We propose an immersive training system grounded in multi-agent large language models (LLMs). Methodologically, we innovatively integrate retrieval-augmented generation (RAG), contextualized prompt engineering, and fine-grained persona modeling to generate dynamic, diverse, and contextually authentic virtual classroom scenarios. These enable teachers to safely practice hate speech detection, escalation prediction of conflicts, and evidence-informed intervention strategies. Key contributions include: (1) the first application of multi-agent LLMs combined with structured persona modeling for anti-hate competency development among educators; (2) empirical insights into the cognitive mechanisms underlying context dependency and annotation disagreement in hate speech identification; and (3) preliminary evaluation results demonstrating significant improvements in trainees’ contextual sensitivity to hate discourse and precision in selecting appropriate interventions.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Computer-aided teacher training is a state-of-the-art method designed to enhance teachers' professional skills effectively while minimising concerns related to costs, time constraints, and geographical limitations. We investigate the potential of large language models (LLMs) in teacher education, using a case of teaching hate incidents management in schools. To this end, we create a multi-agent LLM-based system that mimics realistic situations of hate, using a combination of retrieval-augmented prompting and persona modelling. It is designed to identify and analyse hate speech patterns, predict potential escalation, and propose effective intervention strategies. By integrating persona modelling with agentic LLMs, we create contextually diverse simulations of hate incidents, mimicking real-life situations. The system allows teachers to analyse and understand the dynamics of hate incidents in a safe and controlled environment, providing valuable insights and practical knowledge to manage such situations confidently in real life. Our pilot evaluation demonstrates teachers' enhanced understanding of the nature of annotator disagreements and the role of context in hate speech interpretation, leading to the development of more informed and effective strategies for addressing hate in classroom settings.

Problem

• Develop LLM-based system for teacher hate incident training
• Simulate diverse hate speech scenarios for safe learning
• Enhance teacher strategies for classroom hate speech management

Innovation

• Multi-agent LLM system for realistic hate incident simulations
• Retrieval-augmented prompting and persona modelling integration
• Contextual analysis and intervention strategy prediction for hate speech