A statistical framework for dynamic cognitive diagnosis in digital learning environments

2025-06-17

AI Summary

To address the global lag in students’ reading skill development, this paper proposes a data-driven Bayesian dynamic cognitive diagnosis framework that leverages digital learning platform log data—such as response time, retry count, and mastery item count—to track individual reading skill trajectories in real time. Methodologically, it introduces the first dynamic latent class model that jointly estimates both the Q-matrix and time-varying latent skill profiles, eliminating reliance on expert-defined Q-matrices by integrating constrained latent class structures with individual covariates for temporally sensitive modeling. Empirical evaluation on authentic educational data demonstrates accurate characterization of individual skill profiles and skill–item associations; simulation studies confirm high recovery accuracy and robustness for both Q-matrix estimation and latent profile inference. The core contribution is the first cognitive diagnostic paradigm that is gender- and background-agnostic, requires no prior Q-matrix specification, and enables fine-grained, time-resolved skill assessment.

Technology Category

• Machine Learning: Bayesian Learning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Reading is foundational for educational, employment, and economic outcomes, but a persistent proportion of students globally struggle to develop adequate reading skills. Some countries promote digital tools to support reading development, alongside regular classroom instruction. Such tools generate rich log data capturing students' behaviour and performance. This study proposes a dynamic cognitive diagnostic modeling (CDM) framework based on restricted latent class models to trace students' time-varying skills mastery using log files from digital tools. Unlike traditional CDMs that require expert-defined skill-item mappings (Q-matrix), our approach jointly estimates the Q-matrix and latent skill profiles, integrates log-derived covariates (e.g., reattempts, response times, counts of mastered items) and individual characteristics, and models transitions in mastery using a Bayesian estimation approach. Applied to real-world data, the model demonstrates practical value in educational settings by effectively uncovering individual skill profiles and the skill-item mappings. Simulation studies confirm robust recovery of Q-matrix structures and latent profiles with high accuracy under varied sample sizes, item counts and different sparsity of Q-matrices. The framework offers a data-driven, time-dependent restricted latent class modeling approach to understanding early reading development.

Problem

• Dynamic tracking of students' evolving reading skills using digital log data
• Automated estimation of skill-item mappings without expert-defined Q-matrices
• Integration of behavioral covariates for personalized cognitive diagnosis in education

Innovation

• Dynamic CDM framework estimates Q-matrix and skill profiles
• Integrates log-derived covariates and individual characteristics
• Bayesian approach models time-varying skill mastery transitions