Fighting AI with AI: Leveraging Foundation Models for Assuring AI-Enabled Safety-Critical Systems

2025-11-25

AI Summary

This work addresses core challenges in AI-enabled safety-critical systems: the opacity of deep neural networks (DNNs), ambiguity in natural-language requirements, and the semantic gap between high-level specifications and low-level implementations. We propose REACT–SemaLens, a two-component framework: REACT leverages large language models (LLMs) to perform requirement consistency checking and automated formal specification generation; SemaLens integrates vision-language models (VLMs) to enable semantics-aware testing and runtime monitoring of DNN-based perception systems grounded in human-interpretable concepts. To our knowledge, this is the first framework to systematically bridge the entire pipeline—from informal requirements to formal specifications, verifiable implementations, and semantics-level verification. The approach significantly enhances explainability, test coverage, and verification efficiency of AI components, demonstrating strong practical applicability in aerospace and autonomous driving domains.

Technology Category

• Natural Language Processing: Safety and Robustness
• Computer Vision: Language and Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

The integration of AI components, particularly Deep Neural Networks (DNNs), into safety-critical systems such as aerospace and autonomous vehicles presents fundamental challenges for assurance. The opacity of AI systems, combined with the semantic gap between high-level requirements and low-level network representations, creates barriers to traditional verification approaches. These AI-specific challenges are amplified by longstanding issues in Requirements Engineering, including ambiguity in natural language specifications and scalability bottlenecks in formalization. We propose an approach that leverages AI itself to address these challenges through two complementary components. REACT (Requirements Engineering with AI for Consistency and Testing) employs Large Language Models (LLMs) to bridge the gap between informal natural language requirements and formal specifications, enabling early verification and validation. SemaLens (Semantic Analysis of Visual Perception using large Multi-modal models) utilizes Vision Language Models (VLMs) to reason about, test, and monitor DNN-based perception systems using human-understandable concepts. Together, these components provide a comprehensive pipeline from informal requirements to validated implementations.

Problem

• Assuring AI integration in safety-critical systems
• Bridging semantic gap between requirements and implementations
• Addressing opacity and scalability in AI verification

Innovation

• LLMs bridge informal requirements to formal specifications
• VLMs test DNN perception using human-understandable concepts
• AI pipeline from requirements to validated implementations