AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations

2026-07-28

AI Summary

This study addresses critical pedagogical risks in educational explanations—such as factual inaccuracies, content omissions, misaligned emphasis, inappropriate difficulty levels, and ideological bias—by introducing the first evaluation framework that supports fine-grained, interpretable auditing. Built upon a five-dimensional scoring rubric, the framework integrates a self-hosted Llama 3.1-8B model fine-tuned on the AIriskEval-edu dataset alongside the GPT-5.5 API, enabling real-time assessment through both AI-simulated teacher and human-input modes. Designed for local deployment to ensure data privacy, the system demonstrates superior performance over GPT-5.5 across most evaluation metrics in empirical experiments, offering educational institutions an efficient, private, and scalable solution for content moderation and quality assurance.

Abstract

We present AIriskEval-edu Demo, a platform that audits the pedagogical quality of instructional explanations and provides explainable audit results. The platform evaluates an explanation against a rubric covering five dimensions of pedagogical risk: factual accuracy, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. For each dimension, it returns a binary decision and a confidence score. Detected risks also include a natural-language rationale and, except for Depth and Completeness, a localized evidence span. The platform integrates GPT-5.5 through an external API and a self-hosted Llama 3.1 8B evaluator that runs on consumer-grade GPUs. The local evaluator is fine-tuned on AIriskEval-edu, a dataset of K-12 instructional explanations with risk and explainability annotations. The platform operates in two modes: in AI mode, both evaluators assess stored explanations generated under six simulated teacher profiles, each representing a distinct pedagogical behavior and potential risk; in human mode, the local evaluator audits user-written explanations in real time. The local evaluator outperforms GPT-5.5 on most reported metrics, offering educational institutions a practical way to keep audited content within their own infrastructure.

Problem

• pedagogical risk
• educational explanations
• factual accuracy
• ideological bias
• student-level appropriateness

Innovation

• pedagogical risk auditing
• explainable AI evaluation
• local LLM deployment
• instructional explanation assessment
• educational AI safety