The AI Assessment Sandbox Configurator: A Framework to Support Technical Assessment in AI Regulatory Sandboxes

2026-10-01

AI Summary

This study addresses the fragmentation of testing tools, incomparability of results, and poor reusability within AI regulatory sandboxes by proposing an open-source evaluation framework aligned with the architectural requirements of the EU AI Act. The framework integrates heterogeneous testing tools through a unified data model and a stable plugin API, while establishing a tiered contribution catalog and audience-segmented reporting mechanism to facilitate multi-stakeholder collaboration and standardized outputs. A real-world sandbox pilot validates its effectiveness in data harmonization and report generation, successfully supporting the production of official exit reports. Ultimately, this work provides a reusable technical foundation for AI compliance assessment.

Abstract

The EU's Artificial Intelligence Act requires all Member States to establish AI Regulatory Sandboxes (AIRS) by August 2027: supervised environments bringing together national Competent Authorities, technical experts, and the organisations under assessment. When AIRS engagements include structured technical testing, running such testing at scale demands dedicated infrastructure, yet the tooling ecosystem remains structurally fragmented, with heterogeneous tools producing outputs that are difficult to compare, trace, and reuse. From the procedural conditions of AIRS engagements and the AI Act obligations for high-risk systems, we derive 11 architectural and governance requirements for the infrastructure that operationalises technical testing within an AIRS. In response to these requirements, we introduce the AI Assessment Sandbox Configurator, an open-source framework combining a curated Catalogue of tests and controls accessed through a stable plug-in API, a shared data model that harmonises heterogeneous outputs, role-specific dashboards for multi-disciplinary interpretation, and audience-segmented reporting. We describe the architecture and current release, and report an early-stage pilot that exercised the harmonisation and reporting layers within a live AIRS engagement and contributed to an official Exit Report. We discuss the roadmap, the governance questions raised by the Catalogue's tiered contribution model, and the institutional pathways through which an open-source assessment ecosystem could emerge across Member States.

Problem

• AI Regulatory Sandboxes
• Technical Assessment
• Tooling Fragmentation
• Artificial Intelligence Act
• Infrastructure

Innovation

• AI Regulatory Sandboxes
• Assessment Framework
• Data Harmonisation
• Plugin API
• Open-source Infrastructure