Assessment in Team Problem-Solving Exercises in Computing Education

2026-07-21

AI Summary

This study addresses the challenge of delayed and incomplete evaluation in team tabletop exercises (TTX), which often arises due to the open-ended and complex nature of such tasks, hindering effective assessment of team learning outcomes. To overcome this limitation, the authors propose a novel approach that integrates clustering algorithms with large language models (GPT-4o and GPT-5.2) to enable automated, scalable evaluation of team performance. Leveraging action logs and communication transcripts from 81 multinational participants alongside standardized scoring rubrics, the method demonstrates that clustering is computationally efficient and reliable, while GPT-5.2 significantly outperforms GPT-4o in evaluating team communication with lower error rates. All data, tools, and the complete TTX scenario have been open-sourced and integrated into the INJECT platform to support educational applications.

Abstract

This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents. While assessment is essential for determining how well teams achieve learning objectives, the complex, open-ended nature of TTXs often leads to delayed or incomplete feedback. TTX learning platforms can record teams' actions and communication; yet, leveraging these data to assess performance is underexplored. To address this gap, we compared two post-TTX team assessment methods -- clustering and large language models (LLMs) -- using an original dataset from 81 participants across two countries. We evaluated these methods against instructor-assigned scores based on standardized rubrics. Clustering grouped teams that approached TTX tasks similarly, enabling instructors to deliver faster, targeted feedback to teams within a cluster. This method was valid and reliable, with low computational requirements. LLMs used the standardized rubrics to assess teams' communication. While GPT-4o frequently disagreed with instructor scores, GPT-5.2 demonstrated considerably lower error. The researched methods have been integrated into INJECT, an open-source TTX learning platform, to support scalability and teaching practice. To encourage community adoption, we publicly share all datasets, software tools, and a full-fledged TTX scenario.

Problem

• team assessment
• tabletop exercises
• computing education
• performance feedback
• open-ended tasks

Innovation

• team assessment
• tabletop exercises
• clustering
• large language models
• computing education