Architectural Isolation as a Timing Safety Primitive for Edge AI Medical Devices: Controlled Experimental Evidence on a Shared-Silicon Platform

2026-04-26

AI Summary

This study addresses the critical challenge of ensuring accuracy, stability, and clinical real-time performance in edge AI medical devices, where existing validation methods inadequately cover temporal safety at the inference layer. On an NVIDIA Jetson Orin Nano Super platform, the authors empirically compare the temporal behavior of a MobileNetV2 model executed via TensorRT FP16 on GPU versus ONNX Runtime FP32 on CPU. Despite both configurations achieving zero Software-Triggered Error Rate (STER), the GPU pipeline maintains end-to-end latency consistently below 11 ms—meeting the 10 Hz clinical cycle requirement—whereas the CPU path exhibits 9.8× higher average latency and exceeds timing bounds by 65%. These findings reveal a structural independence among accuracy, stability, and temporal constraints, prompting the proposal of a novel validation paradigm that jointly assesses STER and latency to fulfill FDA regulatory expectations for inference-layer robustness.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

A system can satisfy accuracy-based validation, maintain output stability (Safety-Threshold Exceedance Rate, STER, equal to zero), and still violate timing constraints under deployment load. These are structurally independent properties that current pre-market validation protocols often do not operationalize at the inference layer. This letter demonstrates their independence through a controlled same-hardware experiment: identical MobileNetV2 models are evaluated under identical adversarial load on two execution paths of the same NVIDIA Jetson Orin Nano Super, a dedicated GPU accelerator (TensorRT FP16, half-precision floating point) and a general-purpose CPU (ONNX Runtime FP32, single-precision floating point). Both paths maintain STER = 0; the CPU path (ONNX Runtime FP32) degrades 7.2x under combined load (mean latency 9.8x higher than the GPU path (TensorRT FP16), which maintains latency below 11 ms), breaching the 10 Hz clinical cycle budget by 65%. Joint STER and latency verification is proposed as a candidate method for operationalizing U.S. FDA Draft Guidance FDA-2024-D-4488 robustness requirements at the inference layer, subject to regulatory review and clinical validation.

Problem

• timing safety
• edge AI medical devices
• inference latency
• safety-threshold exceedance
• regulatory validation

Innovation

• architectural isolation
• timing safety
• edge AI medical devices
• latency verification
• Safety-Threshold Exceedance Rate