'A bit of chaos and madness': The AI Assessment Scale and the work of assessment reform

2026-06-25

AI Summary

This study addresses the challenges generative artificial intelligence poses to higher education assessment systems, emphasizing the urgent need for reform that upholds academic integrity, equity, and validity. Employing qualitative methods, it offers the first empirical investigation into the implementation of an Artificial Intelligence Assessment Scale (AIAS) across two institutions—a private international university in Vietnam and a public university in the UK—tracing its journey from policy adoption to classroom practice. Grounded in critical AI literacy theory and analyzed through hybrid thematic analysis of focus group data from 30 instructors across five groups, the research highlights the pivotal roles of institutional context, disciplinary background, and faculty capacity-building in the effective enactment of AIAS. It demonstrates that when detached from learning objectives and disciplinary contexts, AIAS risks becoming performative; however, when appropriately contextualized, it can meaningfully enhance authentic assessment and student engagement.

Abstract

Generative artificial intelligence (GenAI) has intensified pressure on universities to redesign assessment while maintaining integrity, equity, and validity. Structured frameworks such as the Artificial Intelligence Assessment Scale (AIAS) offer one response, but evidence of how staff experience their implementation remains limited. This qualitative study examines AIAS implementation at a private international university in Vietnam and a public university in the United Kingdom. Data from five focus groups with 30 academic staff were analysed using hybrid thematic analysis, with Critical AI Literacy used as a sensitising concept. Six themes were developed: recognising and integrating AI, facilitating conditions, building capacity, pathways to adoption, ethics in practice, and reframing pedagogy. Staff valued the AIAS as a shared language for legitimising GenAI use, clarifying boundaries, and prompting reflection on assessment design. However, implementation was shaped by governance, tool access, staff confidence, workload, integrity concerns, disciplinary context, and alignment with learning outcomes. The findings show that the AIAS could prompt authentic assessment design and student engagement, but may become a compliance layer when disconnected from learning outcomes, disciplinary context, and staff capacity. This study contributes empirical evidence on the institutional conditions through which GenAI assessment frameworks move from policy adoption to pedagogical enactment.

Problem

• generative artificial intelligence
• assessment reform
• AI assessment scale
• academic integrity
• pedagogical enactment

Innovation

• Artificial Intelligence Assessment Scale
• Generative AI
• assessment reform
• Critical AI Literacy
• pedagogical enactment