design assignment scaffolding

Current benchmarks struggle to precisely diagnose the specific skill gaps of large language models in compositional reasoning. This work proposes the Scaffolding-based Task Design (STaD) framework, which, for the first time, integrates educational scaffolding theory into model evaluation. By generating controlled task variants through structured and incremental support, STaD systematically decomposes and probes compositional reasoning capabilities. Operating under a black-box assumption, the approach enables scalable, fine-grained diagnosis of skill deficiencies. Experiments across six models and three reasoning benchmarks reveal distinct and concrete failure patterns unique to each model, demonstrating STaD’s effectiveness and novelty in pinpointing weaknesses in compositional reasoning.

High-quality instructional dialogue data is critically scarce due to privacy sensitivities and the high cost of expert involvement. To address this, we propose an expert-participatory generation framework: leveraging large language models (LLMs) to simulate novice teachers with diverse personality profiles, engaging them in multi-turn, realistic pedagogical dialogues with human education experts; integrating a personality modulation mechanism and real-time expert feedback loops to ensure ecological validity and privacy preservation. Using this framework, we construct a high-fidelity dataset and perform instruction tuning on LLaMA. The resulting expert model significantly outperforms GPT-4o in instructional relevance, cognitive depth, and reflective questioning ability, as validated by expert evaluation. This work introduces the first LLM-driven “personified novice” simulation paradigm—enabling scalable, high-fidelity, and ethically grounded training data infrastructure for educational AI systems.

This study investigates how to effectively support learners in formulating high-quality questions across computational education tasks with varying degrees of openness. To this end, two large language model–based scaffolding systems were designed and pilot-deployed: guided questioning (an indirect scaffold) and worked examples (a direct scaffold), evaluated through a unified framework grounded in Bloom’s taxonomy. The work innovatively compares and integrates these two AI scaffolding modalities, proposing a sequential “reflect-then-exemplify” strategy that balances immediate question quality improvement with deeper cognitive engagement. Findings indicate that direct scaffolding yields more pronounced immediate gains in question quality, whereas indirect scaffolding is more favorably received by students and fosters richer reflection on the process of question design.

This study addresses the challenge of enhancing cross-domain generalization and feedback quality for large language models (LLMs) in automated scoring of formative assessments across multidisciplinary domains—science, computing, and engineering. We propose a novel framework integrating Evidence-Centered Design (ECD), human-AI collaborative closed-loop prompt engineering, and teacher-student feedback-driven active learning to ensure interpretability, robustness, and continuous improvement of scoring systems. Crucially, we unify these three components to enable cross-course transferability and iterative refinement. Evaluated in authentic educational settings using GPT-4 with chain-of-thought reasoning, our approach achieves up to a 24.5% improvement in scoring accuracy over baseline methods. Moreover, teacher–student collaborative feedback significantly enhances inter-rater consistency and the interpretability of generated feedback. The framework provides a scalable, methodology-driven foundation for AI-enhanced educational assessment.

This study addresses the challenges students face in diagnostic reasoning—particularly susceptibility to cognitive biases such as premature closure and overreliance on heuristics, as well as limited strategy transferability—within a situated learning environment for pharmacy technician training. For the first time, it comparatively examines two theory-driven scaffolding dialogue strategies: structured and problematizing. An intelligent tutoring agent, integrating learning analytics and large language models, dynamically intervenes in learners’ trajectories. Findings indicate that both scaffolding approaches effectively promote the use of diagnostic strategies: structured scaffolding enhances the accuracy of active interaction, while problematizing scaffolding fosters constructive engagement. Notably, task complexity exerts a significantly stronger influence on performance than either prior knowledge or scaffolding type.

This study addresses the challenge of effectively guiding students to transition from misconceptions to accurate causal reasoning within a single classroom session, particularly in engineering fault-diagnosis contexts. It proposes a “detective-style scaffolding instructional framework” that reconfigures classroom voting systems into evidence-centered reasoning probes—rather than mere engagement tools—through three stages: hypothesis activation, evidence structuring, and causal integration. The approach exposes how conventional scoring often misjudges reasoning quality and instead employs dual-precision reasoning analysis to assess learning outcomes. In experiments, 80 third-year polymer engineering students increased their correct identification rate of humidity as the root cause from 29% to 100% within 90 minutes; additionally, 26 high school students without engineering backgrounds achieved 100% accuracy on transfer tasks and demonstrated significantly enhanced confidence in data analysis and ability to interpret AI-generated explanations.

Learning analytics systems increasingly integrate large language models (LLMs) to provide adaptive scaffolding in complex learning environments, yet personalization is often driven by global instructional choices rather than principled alignment with learning theory, limiting effectiveness and pedagogical grounding. In prior work, we examined how structuring and problematizing scaffolding approaches can be instantiated through LLM agents in a scenario-based learning environment for diagnostic reasoning. While both approaches supported learning, we observed systematic differences in learner interaction patterns and clear tendencies indicating that different diagnostic strategies benefited from distinct forms of scaffolding. Building on these findings, we propose a theory-informed scaffolding design grounded in the Knowledge Learning Instruction (KLI) framework, as different diagnostic strategies target different types of knowledge and require different instructional mechanisms. We use KLI to guide the alignment between strategy demands and scaffolding approaches and introduce a KLI-informed hybrid LLM agent that adapts its pedagogical support according to the diagnostic strategy being practiced, rather than applying a single global scaffolding approach. We hypothesize that this design could enable better learning gains.

Existing approaches struggle to effectively quantify adaptive scaffolding behaviors in authentic tutoring dialogues, particularly amid the rise of remote human tutoring and large language models. This work proposes the first analytical framework that integrates role-specific semantic alignment with temporal dynamics, modeling semantic relationships among tutor utterances, student responses, problem statements, and correct solutions through embedding-based representation learning. Applying cosine similarity and mixed-effects models to 1,576 mathematics tutoring dialogues, the study reveals that tutors initially focus on problem content, and the degree of semantic alignment between students’ answers and the target solution significantly predicts tutoring progress. These findings demonstrate that scaffolding is a continuous, role-sensitive, and semantically driven process.

This study investigates how large language models integrate instructional cues from system prompts, user prompts, and JSON schemas in structured output tasks, with particular focus on performance degradation when these sources conflict. Through single-field classification experiments across ten models from the GPT and Claude families, the authors conduct ablation studies on instruction placement, conflict scenarios, and interventions involving intermediate reasoning fields. Findings reveal that JSON schemas are not passive metadata but can substantially override prompt-based instructions: schema descriptions alone yield 13 percentage points lower accuracy than system prompts in non-conflicting settings and up to 45 points lower under conflict. Introducing intermediate reasoning fields improves accuracy by 15–24 points, even surpassing prompt-only approaches. These results motivate a unified view of prompts and schemas as a cohesive instruction interface.

This work addresses the scalability bottleneck in AI tutoring systems caused by the labor-intensive, manual construction of structured procedural skill models. To overcome this limitation, the authors propose a human-in-the-loop text-to-model generation approach that leverages large language models to automatically transform instructional texts into procedural skill models conforming to the Task-Method-Knowledge (TMK) ontology. The method integrates ontology-constrained prompting with template-driven generation and incorporates expert validation of causal logic and failure conditions. This framework preserves model structural integrity and semantic alignment while substantially reducing expert modeling effort. Evaluated in a graduate-level AI course, the approach produced 23 skill models with 50–70% less expert time investment, and the generated models demonstrated high reproducibility under fixed inputs.