Can Automated Feedback Turn Students into Happy Prologians?

2025-04-23

AI Summary

In large-scale logic programming courses, delayed personalized feedback impedes student learning. Method: This study designs and implements ProHelp—a Prolog tutoring system that systematically integrates and empirically compares three automated feedback modalities: automated testing, syntactic diagnosis, and execution trace visualization—built upon an extended Prolog interpreter, a static analyzer, and a dynamic tracer. Contribution/Results: A mixed-methods evaluation involving over 1,000 students—including structured surveys and behavioral log analysis—confirms that all feedback types significantly improve learning experience and satisfaction (p < 0.01). Automated testing is rated most effective by students (72.3%). The study further identifies a hierarchy of feedback utility and reveals divergent student priorities regarding explainability, debugging support, and other future features. These findings provide empirical grounding and methodological guidance for designing intelligent programming education systems.

Technology Category

• Knowledge Representation and Reasoning: Logic Programming
• Machine Learning: Statistical Relational/Logic Learning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Giving personalized feedback to students is very important to the learning process. However, doing so in a timely manner can be difficult to accomplish in very large courses. Recent work has explored different types of automated feedback adapted to different languages and programming paradigms, particularly logic programming. In ProHelp, we implemented several of these types of feedback so that they could be used by students enrolled in a logic programming class. Then, we surveyed those students to find if the feedback was useful and which types of feedback they preferred. Results show that students found all types of feedback helpful, with automatic testing, in particular, being the most helpful type. We also explore student preferences for which types of feedback they would most like to see implemented in the future.

Problem

• Providing timely personalized feedback in large courses
• Exploring automated feedback for logic programming students
• Assessing student preferences for feedback types effectiveness

Innovation

• Automated feedback for logic programming students
• Implemented multiple feedback types in ProHelp
• Automatic testing was most helpful feedback