ADMITBench: A Safety-Governed Reference Framework for Evaluating the Admissibility of Industrial LLM Advisories

2026-08-04

AI Summary

This work addresses the lack of systematic safety evaluation mechanisms for action recommendations generated by large language models (LLMs) in industrial settings, where it is challenging to verify whether such suggestions are sufficiently evidence-based, compliant with authorization protocols, and aligned with facility-specific consequence constraints. To bridge this gap, we propose ADMITBench—the first acceptability assessment framework tailored for industrial LLM recommendations—featuring a non-compensatory, explicit safety-checking mechanism. The framework integrates versioned evaluation contracts and plant configuration profiles to establish a three-dimensional verification architecture spanning evidence, permissions, and consequences. We release an open-source implementation (v0.1.0) that provides a context-sensitive, structured, and auditable benchmark for determining the admissibility of LLM-generated recommendations in industrial environments.

Abstract

This white paper presents ADMITBench, a reference framework for evaluating industrial LLM advisories at the level of the proposed action. The framework implements a versioned, safety-governed evaluation contract that checks whether a recommendation is supported by the available evidence, permitted under the stated authority and procedure, and acceptable under the plant-specific consequence checks encoded in the selected evaluation profile. In this report, \emph{safety-governed} means that eligibility is determined through explicit, non-compensatory checks derived from a versioned plant profile; it does not mean that the evaluator, model, or plant has been safety-certified. Release 0.1.0 is a public reference implementation for technical and research evaluation, not an authorisation for physical execution.

Problem

• industrial LLM advisories
• admissibility evaluation
• safety-governed framework
• action-level assessment
• plant-specific constraints

Innovation

• safety-governed evaluation
• industrial LLM advisories
• admissibility framework
• versioned plant profile
• non-compensatory checks