The Missing Layer: Specification Infrastructure for AI Oversight

2026-07-26

AI Summary

This work addresses the absence of standardized, composable oversight infrastructure in current AI deployments, which leads teams to repeatedly build fragmented auditing and monitoring mechanisms. The authors propose a five-layer, six-dimension framework for AI oversight, with a particular focus on formally defining— for the first time—the “normative layer.” This layer translates human intent into executable, traceable, and upgradable machine-checkable norms through six design principles, including elicitable, adversarially aware, and governable specifications. Integrating formal methods, policy languages (e.g., Cedar, OPA), and constitutional AI concepts, the study introduces CARMA, a norm-driven runtime oversight prototype that demonstrates how a single norm can uniformly drive execution, evaluation, and upgrading. The system validates the feasibility of reusing composable oversight components across teams.

Abstract

AI safety has a missing layer. Interpretability, formal methods, security engineering, evaluation methodology, and reinforcement-learning safety each produce substantial work, but the resulting artifacts do not compose into deployable oversight: every team fielding an agentic system builds its own audit schema, policy dialect, monitoring stack, and escalation path, mostly reinventions of patterns understood elsewhere. We diagnose this as a coordination gap, not a research gap, and propose a two-axis taxonomy: five technical layers (Legibility, Specification, Mediation, Evaluation, Escalation) crossed with six concerns spanning alignment, robustness, adversarial defense, security, governance, and accountability, populating the resulting 5x6 matrix with existing work. Layer 2 (Specification), where humans translate intent into machine-checkable artifacts, is the connective tissue every layer depends on, yet it lacks four marks of a mature engineering discipline: shared vocabulary, design principles, composability standards, and governance practices. We propose six design principles for Layer 2, from elicitability and composability to adversary-awareness, traceability, and governability, made concrete through worked examples and a reference architecture turning specifications into runtime enforcement, evaluation, and escalation. Existing systems such as Cedar, Constitutional AI, and Open Policy Agent each address a fragment of Layer 2 well and the matrix poorly; treating them as fragments of one shared layer makes composition tractable. As evidence, we introduce CARMA, a Layer 2 prototype for autonomous ETL agents in which one specification drives enforcement, evaluation, and escalation, with every decision traceable to a versioned specification, naming what AI oversight is missing and giving independent teams principles to build the missing pieces so they compose.

Problem

• Specification Infrastructure
• AI Oversight
• Coordination Gap
• Composability
• Governance

Innovation

• Specification Infrastructure
• AI Oversight
• Composability
• Governability
• CARMA