Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems

2026-05-23

AI Summary

This work addresses the challenge of "silent physical action failures" in embodied AI systems—instances where agents execute actions that appear plausible but are in fact erroneous, evading detection by conventional safety mechanisms during runtime. The paper formally defines this failure mode for the first time and articulates the boundary problem of runtime action authorization. It proposes a unified framework that integrates embodied foundation models, world models, uncertainty estimation, and verification techniques to classify and evaluate protective mechanisms. Through systematic analysis of the limitations inherent in current safety approaches, this study establishes a theoretical foundation and provides a practical evaluation pathway for building trustworthy runtime authorization systems in physical AI.

Abstract

Physical AI systems increasingly map multimodal observations, language instructions, and learned world representations into physically consequential actions. Robotics foundation models, vision-language-action models, and world-model-based autonomous systems can condition decisions that move vehicles, robots, drones, and industrial machines. This transition exposes a safety problem that is not fully captured by conventional AI content moderation or by classical robot safety alone: a black-box model may issue a physically consequential action while appearing confident, plausible, and semantically aligned. The resulting failure can be silent, arising from sensor drift, occlusion, state-estimation error, distribution shift, hallucinated affordances, or invalid physical assumptions before downstream hardware controllers detect a violation. Across embodied foundation models, world models, robotics simulation, embodied safety benchmarks, safe control, runtime assurance, uncertainty estimation, verification, and guardrail evaluation, model capability and safety mechanisms have advanced along largely separate technical tracks. A recurring gap synthesized here is that no single stream surveyed in this review supplies a complete runtime authorization boundary between black-box Physical AI models and physical execution. The resulting analysis develops a bounded problem formulation, a definition of silent physical-action failure, a taxonomy of runtime guardrail functions, and evaluation requirements for comparing guardrails as Physical AI assurance mechanisms.

Problem

• Silent Failures
• Physical AI
• Runtime Authorization
• Autonomous Systems
• Safety Assurance

Innovation

• silent failure
• runtime action authorization
• Physical AI
• guardrail functions
• embodied safety