Synthesizing Efficient and Permissive Programmatic Runtime Shields for Neural Policies

2024-10-08

arXiv.org

AI Summary

Existing runtime safety shields for deploying neural network policies in control systems suffer from high computational overhead and limited fault tolerance. To address this, we propose Aegis—a novel framework that formulates shield synthesis as a sketch-guided program synthesis problem, integrating counterexample-guided inductive synthesis (CEGIS) with Bayesian optimization. Aegis automatically generates lightweight, formally verified safety shields while ensuring rigorous safety guarantees. Evaluated on eight representative control systems, Aegis achieves 100% interception of unsafe commands. Compared to state-of-the-art approaches, it reduces average inference latency by 2.2×, decreases memory footprint by 3.9×, and lowers spurious interventions by 1.5×. These improvements significantly enhance real-time performance and robustness without compromising formal safety assurance.

Technology Category

Philosophy and Ethics of AI: Safety, Robustness & TrustworthinessMultiagent Systems: Adversarial AgentsHumans and AI: Human-Aware Planning and Behavior Prediction

Application Category

Responsible Web: Machine-in-the-loop, human agency and autonomySearch and Retrieval-Augmented AI: Agentic searchSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

With the increasing use of neural policies in control systems, ensuring their safety and reliability has become a critical software engineering task. One prevalent approach to ensuring the safety of neural policies is to deploy programmatic runtime shields alongside them to correct their unsafe commands. However, the programmatic runtime shields synthesized by existing methods are either computationally expensive or insufficiently permissive, resulting in high overhead and unnecessary interventions on the system. To address these challenges, we propose Aegis, a novel framework that synthesizes lightweight and permissive programmatic runtime shields for neural policies. Aegis achieves this by formulating the seeking of a runtime shield as a sketch-based program synthesis problem and proposing a novel method that leverages counterexample-guided inductive synthesis and Bayesian optimization to solve it. To evaluate Aegis and its synthesized shields, we use eight representative control systems and compare Aegis with the current state-of-the-art. Our results show that the programmatic runtime shields synthesized by Aegis can correct all unsafe commands from neural policies, ensuring that the systems do not violate any desired safety properties at all times. Compared to the current state-of-the-art, Aegis's shields exhibit a 2.2$ imes$ reduction in time overhead and a 3.9$ imes$ reduction in memory usage, suggesting that they are much more lightweight. Moreover, Aegis's shields incur an average of 1.5$ imes$ fewer interventions than other shields, showing better permissiveness.

Problem

• Ensuring safety of neural policies in control systems
• Reducing computational overhead of runtime shields
• Improving permissiveness to minimize unnecessary interventions

Innovation

• Lightweight programmatic runtime shields synthesis
• Counterexample-guided inductive synthesis method
• Bayesian optimization for shield permissiveness