Mining Hierarchies with Conviction: Constructing the CS1 Skill Hierarchy with Pairwise Comparisons over Skill Distributions

2024-10-16

arXiv.org

AI Summary

This study investigates prerequisite relationships among foundational programming skills in CS1 to inform data-driven curriculum design. Leveraging exam data from over 600 students across four assessments, we propose the first application of the conviction metric from association rule mining to model skill hierarchies in computing education. We infer pairwise prerequisites among five core skills: Write, Trace, Reverse Trace, Sequence, and Explain. Results reveal that Tracing is a robust prerequisite for Writing; Writing→Explain constitutes a newly identified prerequisite link (valid under mean-based thresholding); and prerequisite dependencies are highly threshold-sensitive—no stable prerequisites emerge between Sequence and either Writing or Explain. Beyond challenging prevailing pedagogical assumptions, this work introduces a reproducible, threshold-aware skill ordering framework. It provides empirical grounding for adaptive learning pathways and layered instructional design in introductory programming.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Reasoning under Uncertainty: Probabilistic Programming
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Security and Privacy: Data transparency and provenance

Abstract

Background and Context: Some skills taught in introductory programming courses are categorized into 1) explaining code, 2) arranging lines of code in correct sequence, 3) tracing through the execution of a program, and 4) writing code from scratch. Objective: Knowing if a programming skill is a prerequisite to another would benefit teachers in properly planning the course and structuring the order in which they present activities relating to new content. Prior attempts to establish a skill hierarchy have suffered from methodological issues. Method: In this study, we used the conviction measure from association rule mining to perform pair-wise comparisons of five skills: Write, Trace, Reverse trace, Sequence, and Explain code. We used the data from four exams with more than 600 participants where students solved programming assignments of different skills for several programming topics. Findings: Our findings matched the previous finding that tracing is a prerequisite for students to learn to write code. Contradicting the previous claims, our analysis showed that using the mean threshold writing code is a prerequisite to explaining code. However, there is no clear relationship when we change the threshold to the median. Unlike prior work, we did not find a clear prerequisite relationship between sequencing code and writing or explaining code. Implications: Our research can help instructors by systematically arranging the skills students exercise when encountering a new topic. The goal is to help instructors properly teach and assess programming in a fashion most effective for learning by leveraging the relationship between skills.

Problem

• Establishing prerequisite relationships among programming skills
• Using conviction measure for pairwise skill comparisons
• Improving course planning and skill hierarchy construction

Innovation

• Pairwise comparisons of skills
• Conviction measure from association rule mining
• Data from exams with 600 participants