🤖 AI Summary
This study addresses the lack of a standardized framework for assessing computational thinking by introducing the Common European Framework of Reference for Languages (CEFR) into the evaluation of Scratch programming proficiency. Leveraging Dr.Scratch, the authors applied fuzzy C-means clustering to over two million projects and mapped the results ordinally onto the CEFR’s A1–C2 scale, establishing a reproducible and transparent diagnostic and progression-tracking system. The work innovatively proposes mechanisms for identifying transitional proficiency stages, quantifying classification confidence, and triggering human-in-the-loop interventions. It reveals a prevalent “B2 bottleneck” in educational settings—only 13.3% of learners attain this level—attributed to difficulties in integrating logical synchronization and data representation, thereby providing empirical grounding for personalized instruction and automated feedback systems.
📝 Abstract
Context: Schools, training platforms, and technology firms increasingly need to assess programming proficiency at scale with transparent, reproducible methods that support personalized learning pathways. Objective: This study introduces a pedagogical framework for Scratch project assessment, aligned with the Common European Framework of Reference (CEFR), providing universal competency levels for students and teachers alongside actionable insights for curriculum design. Method: We apply Fuzzy C-Means clustering to 2008246 Scratch projects evaluated via Dr.Scratch, implementing an ordinal criterion to map clusters to CEFR levels (A1-C2), and introducing enhanced classification metrics that identify transitional learners, enable continuous progress tracking, and quantify classification certainty to balance automated feedback with instructor review. Impact: The framework enables diagnosis of systemic curriculum gaps-notably a "B2 bottleneck" where only 13.3% of learners reside due to the cognitive load of integrating Logic Synchronization, and Data Representation--while providing certainty--based triggers for human intervention.