Making the Invisible Visible: A Framework for Reflective AI Use in Software Engineering Education

2026-09-28

AI Summary

This study addresses the opacity of generative AI interactions in software engineering education, which impedes the assessment of students’ critical judgment and responsible use capabilities. To overcome this challenge, the authors propose an “AI Log” framework that integrates execution tracing with cognitive auditing techniques to systematically document prompts, verification strategies, and reflective processes. Furthermore, they construct a lightweight, model-agnostic reflective framework that shifts the evaluative focus from final artifacts toward the processes of verification, intervention, and evaluative judgment. This approach reveals critical learning moments during students’ context adaptation, effectively enhancing both the transparency and the evaluative depth of AI-assisted learning environments.

Abstract

Generative AI (GenAI) is increasingly embedded in software engineering education, supporting activities such as requirements development, design exploration, documentation, and prototyping. However, educators often have visibility only into final artefacts, with limited insight into how students evaluate, verify, and refine AI-generated outputs during the learning process. This creates challenges for assessing evaluative judgement and responsible AI-assisted practice. This paper introduces the AI Journal, a structured reflection framework designed to make student-GenAI interaction visible in first-year software engineering education. The framework combines execution tracking, which records prompts, outputs, intent, and interaction context, with cognitive auditing, which captures verification strategies, intervention decisions, confidence judgements, critical learning moments, and reflections on AI-supported work. Deployed in a first-semester software engineering course, the AI Journal enabled visibility into aspects of student learning not observable through artefact-based assessments alone. Preliminary observations suggested variation in verification practices, intervention strategies, and perceptions of AI-supported work. Critical learning moments frequently occurred when students evaluated contextual suitability, feasibility, and requirements alignment rather than identifying obvious errors. The AI Journal demonstrates a practical, lightweight, and model-agnostic approach for making AI-assisted learning processes visible. By foregrounding verification, intervention, and reflection, it shifts attention from product-focused assessment toward evaluative judgement and responsible AI-assisted practice.

Problem

• Generative AI
• Software Engineering Education
• Assessment Visibility
• Evaluative Judgement
• Responsible AI Use

Innovation

• Generative AI
• Reflective Framework
• Cognitive Auditing
• Execution Tracking
• Software Engineering Education