Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics

2026-05-31

AI Summary

This study addresses the challenge of moving undergraduate students beyond using large language models (LLMs) merely as answer-providing tools in statistical learning, aiming instead to foster independent reasoning and deep conceptual understanding. Through a quasi-experimental design in an introductory probability and statistics course, the research compares three conditions: no LLM access, unrestricted LLM use, and guided LLM interaction. It proposes and evaluates a “guided scaffolding” framework that leverages a unified LLM platform enhanced with step-by-step prompting, answer verification, ethical guidelines, and training in reasoning-oriented help-seeking. Findings indicate that students in the guided condition outperformed peers on unaided assessments, demonstrating superior problem-solving independence and more accurate self-assessment calibration, thereby substantiating that high-quality, structured LLM interactions can significantly enhance long-term learning outcomes.

Abstract

Large language models (LLMs) are increasingly entering students' learning practices, but their educational value depends on whether they support reasoning or enable task completion without engagement. This study examines guided LLM use in an undergraduate Probability and Statistics course, focusing on the gap between assigned access and actual interaction quality. In a four-week quasi-experimental summer program, students were organized into three balanced conditions: no LLM access, unrestricted LLM access, and guided LLM access. The guided condition used the same LLM platform as the unrestricted condition, but students received explicit training and rules promoting reasoning-focused help-seeking, stepwise hints, verification, and ethical use. All quizzes and the delayed final exam were completed without LLM or external assistance, allowing us to distinguish AI-supported practice performance from independent learning. Results show that guided use was associated with clearer learning-oriented interaction patterns than unrestricted access, especially in prioritizing reasoning over final answers and requesting stepwise support. Guided-LLM students showed stronger no-help quiz performance during the intervention phase, whereas unrestricted access appeared more useful for assisted practice completion than for consistently improving independent performance. Available time measures did not support a simple duration-based explanation, and self-assessment calibration suggested better alignment between perceived and demonstrated understanding in the Guided-LLM condition. Overall, LLM access alone appears to be an incomplete educational intervention. For Artificial Intelligence in Education (AIED), the central design challenge is to scaffold how students use LLMs so that these systems function as partners in reasoning rather than answer-getting tools.

Problem

• large language models
• independent learning
• reasoning
• scaffolding
• AI in education

Innovation

• guided LLM scaffolding
• reasoning-focused interaction
• independent learning
• AI in Education
• stepwise hints