Detection of Disengagement from Voluntary Quizzes: An Explainable Machine Learning Approach in Higher Distance Education

2025-07-03

AI Summary

This study addresses the problem of latent academic disengagement—specifically, student withdrawal from non-mandatory quizzes—in remote higher education. We propose an explainable machine learning framework for early identification, leveraging log data from 42 Moodle-based courses. A multi-model comparative architecture incorporating SHAP (Shapley Additive Explanations) systematically evaluates eight algorithms and interprets salient behavioral features—including quiz initiation delay and interaction sparsity. The best-performing model achieves a balanced accuracy of 91% and a recall rate exceeding 85% for disengaged students. Our primary contribution is the first deep integration of SHAP into the academic disengagement detection pipeline for distance education, jointly optimizing predictive performance and decision interpretability. This yields actionable, pedagogically meaningful behavioral insights for educators, thereby enhancing both the feasibility and effectiveness of proactive academic support interventions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Students disengaging from their tasks can have serious long-term consequences, including academic drop-out. This is particularly relevant for students in distance education. One way to measure the level of disengagement in distance education is to observe participation in non-mandatory exercises in different online courses. In this paper, we detect student disengagement in the non-mandatory quizzes of 42 courses in four semesters from a distance-based university. We carefully identified the most informative student log data that could be extracted and processed from Moodle. Then, eight machine learning algorithms were trained and compared to obtain the highest possible prediction accuracy. Using the SHAP method, we developed an explainable machine learning framework that allows practitioners to better understand the decisions of the trained algorithm. The experimental results show a balanced accuracy of 91%, where about 85% of disengaged students were correctly detected. On top of the highly predictive performance and explainable framework, we provide a discussion on how to design a timely intervention to minimise disengagement from voluntary tasks in online learning.

Problem

• Detect student disengagement in non-mandatory quizzes
• Identify informative student log data from Moodle
• Develop explainable ML framework for intervention strategies

Innovation

• Explainable ML framework with SHAP method
• Eight ML algorithms for high accuracy
• Moodle log data for disengagement detection