Of Good Demons and Bad Angels: Guaranteeing Safe Control under Finite Precision

2025-07-30

AI Summary

Safety-critical neural network control systems (NNCSs) deployed with finite-precision arithmetic suffer from rounding errors, causing a critical gap between theoretical safety guarantees and actual runtime behavior. Method: We propose the first formal verification method that embeds finite-precision robustness into differential dynamic logic (dL), via a “good demon–bad angel” hybrid game model unifying bounded disturbances across perception, computation, and actuation. Our approach jointly optimizes mixed-precision fixed-point implementations and certifies neural network robustness, enabling end-to-end closed-loop verification from formal specifications to deployable code. Contribution/Results: The method delivers efficient, formally proven infinite-horizon safety guarantees on automotive and avionic case studies, bridging a fundamental gap between dL-based deductive verification and real-world embedded implementation.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

As neural networks (NNs) become increasingly prevalent in safety-critical neural network-controlled cyber-physical systems (NNCSs), formally guaranteeing their safety becomes crucial. For these systems, safety must be ensured throughout their entire operation, necessitating infinite-time horizon verification. To verify the infinite-time horizon safety of NNCSs, recent approaches leverage Differential Dynamic Logic (dL). However, these dL-based guarantees rely on idealized, real-valued NN semantics and fail to account for roundoff errors introduced by finite-precision implementations. This paper bridges the gap between theoretical guarantees and real-world implementations by incorporating robustness under finite-precision perturbations -- in sensing, actuation, and computation -- into the safety verification. We model the problem as a hybrid game between a good Demon, responsible for control actions, and a bad Angel, introducing perturbations. This formulation enables formal proofs of robustness w.r.t. a given (bounded) perturbation. Leveraging this bound, we employ state-of-the-art mixed-precision fixed-point tuners to synthesize sound and efficient implementations, thus providing a complete end-to-end solution. We evaluate our approach on case studies from the automotive and aeronautics domains, producing efficient NN implementations with rigorous infinite-time horizon safety guarantees.

Problem

• Ensuring neural network safety in cyber-physical systems
• Addressing finite-precision roundoff errors in verification
• Providing infinite-time horizon safety guarantees

Innovation

• Incorporates finite-precision robustness into safety verification
• Models control as hybrid game between Demon and Angel
• Uses mixed-precision fixed-point tuners for implementation