ParaStudent: Generating and Evaluating Realistic Student Code by Teaching LLMs to Struggle

2025-07-16

AI Summary

This study investigates whether large language models (LLMs) can generate “student-like” code—exhibiting authentic characteristics such as common errors, iterative refinement, stylistic diversity, and temporal evolution. To this end, we propose the first systematic framework for modeling student programming evolution, grounded in multi-semester, timestamped student submission data. Our approach integrates context-aware prompting, temporal modeling, and LLM fine-tuning. We further design a dual-granularity evaluation framework: fine-grained (semantic, functional, and stylistic fidelity) and coarse-grained (learning trajectory and error distribution). Crucially, our method explicitly models the “struggling learning” process—not merely optimal solutions. Experiments demonstrate significant improvements in the authenticity of generated code regarding error patterns, revision pacing, and stylistic variation, validating that explicit modeling of learning dynamics is essential for producing credible student code.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) have shown strong performance on programming tasks, but can they generate student-like code like real students - imperfect, iterative, and stylistically diverse? We present ParaStudent, a systematic study of LLM-based "student-like" code generation in an introductory programming course setting. Using a dataset of timestamped student submissions across multiple semesters, we design low- and high-resolution experiments to model student progress and evaluate code outputs along semantic, functional, and stylistic dimensions. Our results show that fine-tuning significantly improves alignment with real student trajectories and captures error patterns, incremental improvements, and stylistic variations more faithfully. This study shows that modeling realistic student code requires capturing learning dynamics through context-aware generation, temporal modeling, and multi-dimensional evaluation. Code for experiments and evaluation is available at href{https://github.com/mmiroyan/ParaStudent}{ exttt{github.com/mmiroyan/ParaStudent}}.

Problem

• Generating imperfect student-like code using LLMs
• Evaluating code on semantic, functional, stylistic dimensions
• Modeling student learning dynamics and error patterns

Innovation

• Fine-tuning LLMs for realistic student code
• Context-aware generation with temporal modeling
• Multi-dimensional semantic functional stylistic evaluation