AgentForge: An Immersive Role-Playing Platform for Learning Agentic Software Engineering

2026-08-04

AI Summary

This study addresses the challenges novice developers face in understanding AI decisions, collaborating effectively, and critically evaluating AI contributions in software engineering. To support learning in authentic engineering contexts, this work proposes an immersive role-playing platform that uniquely integrates multi-agent software engineering with pedagogical role-play: learners assume one of four roles—task planner, patch writer, code reviewer, or test executor—while AI agents fulfill the remaining roles. The platform scaffolds learning through role-specific guidance, metacognitive support, and visualizations of agent collaboration processes and intermediate artifacts. An empirical evaluation with 37 novices demonstrates high task completion rates and significant improvements in understanding code repair and AI collaboration (p_adj < .001). Notably, although the code reviewer role proved most challenging, it yielded particularly pronounced learning gains.

Abstract

Agentic AI is increasingly used to coordinate planning, implementation, review, and testing in software development, yet it often offers limited transparency into its decisions and interactions. Many such systems also assume that users can effectively guide the AI's decisions and validate its outputs. This assumption poses a particular challenge for novices, who must simultaneously learn how agentic AI works, how to collaborate with it effectively, and how to evaluate its outputs critically. To address this challenge, we present \textit{AgentForge}, an immersive learning system in which novices take on one of four software-engineering roles: Task Planner, Patch Author, Code Reviewer, or Test Runner, within a multi-agent code-repair workflow. In each practice session, the novices perform their chosen role while AI agents perform the remaining three. Through role-based scaffolding and metacognitive support, AgentForge clarifies role-specific responsibilities, makes agent coordination and intermediate artifacts visible, and encourages novices to monitor and evaluate their decisions. In a study with 37 novice developers, participants achieved high task-completion rates with AI-agent support. However, interaction demands differed significantly across practices: the Code Reviewer practice required more interaction turns, reroutes, and completion time ($p_{\mathrm{adj}} = .004$) and was perceived as the most challenging. Participants nevertheless reported significant gains in their understanding of software repair and agent collaboration ($p_{\mathrm{adj}} < .001$). These findings suggest that AgentForge can help novices develop practical software-engineering skills while learning to collaborate with agentic AI more critically and effectively.

Problem

• Agentic AI
• software engineering education
• AI transparency
• novice developers
• human-AI collaboration

Innovation

• Agentic AI
• Immersive Learning
• Role-Based Scaffolding
• Multi-Agent Collaboration
• Metacognitive Support