NLP-Informed Dynamic Cognitive Diagnosis Modelling

2026-04-08

AI Summary

This study addresses the challenge of inaccurate Q-matrix estimation in traditional cognitive diagnostic models under sparse response data, which often leads to ambiguous item-to-skill mappings. To overcome this limitation, the authors propose a dynamic cognitive diagnosis model that integrates semantic information from item texts. Specifically, for the first time, semantic representations of items and their options—extracted via natural language processing—are incorporated as Bayesian priors over the Q-matrix. Within a unified framework, the model jointly infers students’ skill proficiencies, item parameters, and skill development trajectories. This data-driven approach to modeling item-skill relationships significantly improves Q-matrix recovery accuracy and enhances parameter estimation performance compared to baseline models that ignore textual information, with particularly pronounced gains in low-data regimes, as demonstrated on the Boost Reading dataset.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Digital learning platforms are increasingly used to support reading development while generating rich log files and item-level textual content. Using these data, this study proposes a dynamic cognitive diagnostic modelling (CDM) framework that incorporates text-derived semantic information to inform the estimation of the Q-matrix. We construct item-level semantic representations of question text and response options, and use these representations to define an informative prior on the Q-matrix. This approach treats text-derived signals as proxies for item complexity and cognitive demands, guiding the item-skill mapping in a data-driven manner. The proposed framework jointly estimates latent skill mastery profiles, item parameters, and transition dynamics over time within a Bayesian framework. We apply the model to data from Boost Reading, a digital reading supplement, focusing on students' vocabulary and comprehension skill development. We compare the proposed framework with a baseline model without any text information and show that the text-derived prior can improve Q-matrix recovery, particularly in settings where response data alone provide limited identification, as well as other model parameters for varying scenarios. This study provides a novel integration of natural language processing and dynamic CDMs, offering a data-driven approach to modelling skill acquisition and item-skill relationships in digital learning environments.

Problem

• cognitive diagnosis modelling
• Q-matrix estimation
• natural language processing
• digital learning
• item-skill mapping

Innovation

• cognitive diagnostic modeling
• natural language processing
• Q-matrix estimation
• Bayesian inference
• digital learning