curriculum alignment measurement

This work proposes an automated classification approach that integrates traditional natural language processing techniques—such as syntactic parsing, part-of-speech tagging, and text embeddings—with large language models to efficiently align computer science course materials with the ACM/IEEE curriculum guidelines. Manual evaluation of such alignment is time-consuming and cognitively demanding; in contrast, the proposed method enables precise semantic-level categorization of instructional documents, significantly enhancing the efficiency of curriculum audits. By automating the mapping between course content and internationally recognized CS education standards, this approach offers a scalable technical solution to support quality assurance in computing education.

This study addresses the lack of reliable methods for evaluating how undergraduate computer science curricula align with international teaching guidelines such as CS2013 and CS2023, and how this alignment evolves across guideline revisions. The authors propose a human-in-the-loop analytical pipeline that structures course content and guideline knowledge units, employs semantic retrieval—incorporating reciprocal rank fusion and lightweight sentence embedding models—to generate matching candidates, and applies clearly defined coverage criteria for human validation, supplemented by Cohen’s kappa to assess inter-rater consistency. For the first time, the approach enables longitudinal measurement of curriculum alignment across three dimensions: topic coverage, competency expression, and cognitive depth, distinguishing structural gaps from changes due to updated standards. Empirical results reveal knowledge unit coverage rates of 50.9% for CS2013 and 49.7% for CS2023, competency coverage around 88%, but a notable decline in adherence to recommended cognitive depth—from 95% to 76%.

This study addresses the challenge of efficiently and transparently aligning learning resources to structured competency frameworks. The authors propose an end-to-end alignment pipeline that first segments instructional and assessment content into granular fragments, then enhances candidate competency retrieval using a competency knowledge graph. A large language model (LLM) selects the most relevant competencies and generates traceable textual evidence, followed by graph-constrained inference to refine predictions. Integrating LLM reasoning, BM25 retrieval, and graph-based constraints, the method substantially outperforms zero-shot, few-shot, and traditional supervised baselines on the UTC Computer Science dataset, achieving a fragment-level micro F1 of 0.57, macro F1 of 0.50, resource-level macro F1 of 0.51, and MRR of 0.82, while producing auditable alignment evidence.

This study systematically evaluates the reliability of large language models (LLMs) for skill extraction in curricular analytics, focusing on their capacity to process large-scale, unstructured course texts. We benchmark four approaches—retrieval-augmented generation (RAG), zero-shot prompting, TF-IDF matching, and BERT embedding similarity—on a corpus of 400 multi-source course documents. To ensure rigorous evaluation, we introduce the first human-in-the-loop assessment framework for this task. Results show that RAG consistently outperforms zero-shot prompting and traditional NLP methods across all document types; zero-shot prompting exhibits poor generalization; and both model selection and prompt engineering significantly impact extraction quality. This work represents the first systematic application of RAG to curricular analytics and empirically validates the efficacy of human-in-the-loop evaluation. It establishes a reproducible methodological foundation and provides empirical evidence to advance intelligent, data-driven educational analysis.

This study addresses the gap between existing Bodies of Knowledge (BoKs) in computing and their effective translation into assessable, competency-oriented curricula. The authors propose an innovative competency-mapping methodology that systematically aligns BoKs with a structured competency framework, resulting in a five-year engineering curriculum encompassing 23 core competencies, organized into five thematic modules and three specialization tracks, and integrating mandatory work-integrated learning projects. To support explicit linkage, collaborative maintenance, and continuous evolution of knowledge-to-competency mappings, the authors developed ISANUMpedia—a semantic web–based collaborative platform. Implemented in the ISANUM engineering degree program, this approach successfully mapped 494 knowledge topics to the 23 competencies, significantly enhancing the curriculum’s professional relevance and assessability.

This study addresses the challenge that existing NLP methods struggle to reliably extract implicit competencies from educational and labor market texts due to the absence of a unified terminology framework and robust credibility assessment. To overcome this, the authors propose a four-stage framework comprising competency formalization via JSON Schema, constrained-prompt dual-model LLM extraction, semantic alignment with the ESCO taxonomy, and a two-tier arbitration mechanism. The approach is rigorously validated through multidimensional evaluation metrics, including Cohen’s kappa (0.79), schema compliance, and document completeness. Applied to computer science curricula at a UAE university, the method extracted 400 competencies, revealing a 25.0% gap in generic skill supply versus demand, compared to only 1.8% in artificial intelligence—demonstrating the framework’s effectiveness, interpretability, and cross-domain applicability.

This work addresses the limitations of traditional programming instruction, which relies on static example repositories and struggles to provide personalized feedback on students’ logical errors. The authors propose a knowledge component (KC)-driven generative approach that first extracts structured KC patterns from the abstract syntax trees of student-submitted code and then leverages these patterns to conditionally guide a generative model in synthesizing problem-solving examples highly relevant to the students’ specific errors. This study presents the first integration of code-pattern-based KCs into generative modeling, establishing an end-to-end pipeline for personalized educational content generation. Expert evaluations demonstrate that the generated examples significantly outperform baseline methods in both topical focus and relevance to student errors.

This study addresses the challenge of mapping cybersecurity course keywords to the Cybersecurity Body of Knowledge (CyBOK) due to ambiguous, overly broad terminology and incomplete alignment. To overcome this, the authors propose an interpretable human-in-the-loop retrieval framework that employs multi-level semantic strategies—including query normalization, manual term expansion, concept weight enhancement, enriched topic descriptions, and domain-sensitive ranking—to generate Top-k candidate CyBOK entries for expert review, thereby avoiding reliance on strict exact matching. The work introduces the ECA-5 evaluation metric, which assesses mapping utility based on expert judgment. Experimental results demonstrate that the proposed approach achieves a 98.00% ECA-5 accuracy on the validation set, significantly outperforming purely structural matching methods and effectively supporting experts in performing efficient and reliable knowledge alignment.

This work addresses the lack of traceability in existing domain-specific fine-tuning approaches, which often leads to blind and inefficient data augmentation. The authors propose a “programming with data” paradigm that treats structured knowledge representations as a unified foundation for both training and evaluation, drawing an analogy to software development: training data serve as source code, model training as compilation, evaluation as unit testing, and data refinement as debugging. This framework enables precise, concept- and reasoning-chain–oriented model repair through structured knowledge extraction, test-driven data engineering, concept-level gap analysis, and diagnosis of broken reasoning chains. Validated across 16 disciplines, the approach significantly enhances model performance without compromising general capabilities, and the authors release an open-source knowledge base, evaluation suite, and training corpora to support reproducibility and further research.