The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment

2024-03-15

Australasian Journal of Educational Technology

AI Summary

The rapid adoption of generative AI (GenAI) in higher education has precipitated academic integrity crises and assessment invalidity, rendering prohibition- or punishment-based policies ineffective. To address this, we propose the five-level dynamic AI-Integrated Assessment Schema (AIAS), a pedagogically grounded framework that systematically guides educators in embedding GenAI organically into assessment design—shifting from deterrence to a “GenAI-for-learning” paradigm centered on human critical thinking and creativity. AIAS integrates evidence-based instructional design principles, GenAI-tool orchestration protocols, and multimodal artifact evaluation methodologies. Pilot implementations across diverse disciplines demonstrate statistically significant reductions in academic misconduct, marked increases in faculty and student GenAI engagement, and substantial growth in innovative outputs (e.g., video essays, interactive reports). Critically, it catalyzes substantive pedagogical transformation. As the first operationalizable, scalable framework for GenAI-integrated assessment in higher education, AIAS offers a globally applicable solution to sustain academic integrity while harnessing GenAI’s transformative potential.

Technology Category

• Philosophy and Ethics of AI: Artificial General Intelligence
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Intelligent Robots: Embodied AI

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The rapid adoption of generative artificial intelligence (GenAI) technologies in higher education has raised concerns about academic integrity, assessment practices and student learning. Banning or blocking GenAI tools has proven ineffective, and punitive approaches ignore the potential benefits of these technologies. As a result, assessment reform has become a pressing topic in the GenAI era. This paper presents the findings of a pilot study conducted at British University Vietnam exploring the implementation of the Artificial Intelligence Assessment Scale (AIAS), a flexible framework for incorporating GenAI into educational assessments. The AIAS consists of five levels, ranging from “no AI” to “full AI,” enabling educators to design assessments that focus on areas requiring human input and critical thinking. The pilot study results indicate a significant reduction in academic misconduct cases related to GenAI and enhanced student engagement with GenAI technology. The AIAS facilitated a shift in pedagogical practices, with faculty members incorporating GenAI tools into their modules and students producing innovative multimodal submissions. The findings suggest that the AIAS can support the effective integration of GenAI in higher education, promoting academic integrity while leveraging technology’s potential to enhance learning experiences.   Implications for practice or policy: Higher education institutions should adopt flexible frameworks like the AIAS to guide ethical integration of GenAI into assessment practices. Educators should design assessments that leverage GenAI capabilities, while supporting critical thinking and human input. Institutional policies related to GenAI should be developed in consultation with stakeholders and regularly updated to keep pace with technological advancements. Policymakers should prioritise research funding into the impacts of GenAI on higher education to inform evidence-based practices.

Problem

• Integrating GenAI in education
• Enhancing academic integrity
• Improving student learning outcomes

Innovation

• Generative AI integration
• AI Assessment Scale
• Academic integrity enhancement