Exploring Heterogeneity and Uncertainty for Graph-based Cognitive Diagnosis Models in Intelligent Education

2024-02-15

AI Summary

In intelligent education, cognitive diagnosis graph models suffer from inaccurate assessment due to unaddressed edge heterogeneity (mixing correct and incorrect responses) and response uncertainty (e.g., guessing, carelessness). To address this, we propose a semantic-aware graph neural network integrated with an information bottleneck-driven edge filtering framework. Our approach is the first to jointly model edge heterogeneity and suppress response uncertainty within a unified information bottleneck framework. We design an unsupervised reliable subgraph learning mechanism and theoretically prove its equivalence to joint optimization of cross-entropy loss and the Hilbert–Schmidt Independence Criterion (HSIC). Leveraging mutual information maximization/minimization and alternating optimization, our method achieves an average 3.2% improvement in diagnostic accuracy across three real-world educational datasets, significantly outperforming state-of-the-art methods. Empirical results validate that disentangling heterogeneous edges and filtering uncertain ones are critical for enhancing cognitive diagnosis performance.

Technology Category

• Reasoning under Uncertainty: Graphical Models
• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph-based Cognitive Diagnosis (CD) has attracted much research interest due to its strong ability on inferring students' proficiency levels on knowledge concepts. While graph-based CD models have demonstrated remarkable performance, we contend that they still cannot achieve optimal performance due to the neglect of edge heterogeneity and uncertainty. Edges involve both correct and incorrect response logs, indicating heterogeneity. Meanwhile, a response log can have uncertain semantic meanings, e.g., a correct log can indicate true mastery or fortunate guessing, and a wrong log can indicate a lack of understanding or a careless mistake. In this paper, we propose an Informative Semantic-aware Graph-based Cognitive Diagnosis model (ISG-CD), which focuses on how to utilize the heterogeneous graph in CD and minimize effects of uncertain edges. Specifically, to explore heterogeneity, we propose a semantic-aware graph neural networks based CD model. To minimize effects of edge uncertainty, we propose an Informative Edge Differentiation layer from an information bottleneck perspective, which suggests keeping a minimal yet sufficient reliable graph for CD in an unsupervised way. We formulate this process as maximizing mutual information between the reliable graph and response logs, while minimizing mutual information between the reliable graph and the original graph. After that, we prove that mutual information maximization can be theoretically converted to the classic binary cross entropy loss function, while minimizing mutual information can be realized by the Hilbert-Schmidt Independence Criterion. Finally, we adopt an alternating training strategy for optimizing learnable parameters of both the semantic-aware graph neural networks based CD model and the edge differentiation layer. Extensive experiments on three real-world datasets have demonstrated the effectiveness of ISG-CD.

Problem

• Intelligent Education
• Cognitive Diagnostic Models
• Complexity and Uncertainty

Innovation

• ISG-CD Model
• Graph-based Knowledge Diagnosis
• Reliable Information Identification