BURNS: Backward Underapproximate Reachability for Neural-Feedback-Loop Systems

2025-05-06

AI Summary

Formal safety verification tools are lacking for neural feedback closed-loop systems. Method: This paper proposes a backward under-approximation method for reachable sets of nonlinear discrete-time systems. It constructs an over-approximation model of system dynamics and reformulates reachable set computation as a tractable mixed-integer linear programming (MILP) problem. Contribution/Results: The approach enables, for the first time, rigorous backward under-approximation of nonlinear discrete closed-loop systems incorporating neural network controllers. We prove the algorithm’s completeness, guaranteeing that the computed set strictly contains the true backward reachable set. Numerical experiments demonstrate its effectiveness in verifying goal-achievement properties, significantly expanding the scope of safety properties amenable to formal verification in learning-based control systems.

Technology Category

• Multiagent Systems: Adversarial Agents
• Reasoning under Uncertainty: Stochastic Optimization
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Learning-enabled planning and control algorithms are increasingly popular, but they often lack rigorous guarantees of performance or safety. We introduce an algorithm for computing underapproximate backward reachable sets of nonlinear discrete time neural feedback loops. We then use the backward reachable sets to check goal-reaching properties. Our algorithm is based on overapproximating the system dynamics function to enable computation of underapproximate backward reachable sets through solutions of mixed-integer linear programs. We rigorously analyze the soundness of our algorithm and demonstrate it on a numerical example. Our work expands the class of properties that can be verified for learning-enabled systems.

Problem

• Verifying goal-reaching properties in neural feedback loops
• Computing underapproximate backward reachable sets for nonlinear systems
• Ensuring safety and performance in learning-enabled control systems

Innovation

• Computes underapproximate backward reachable sets
• Uses mixed-integer linear programs solutions
• Overapproximates system dynamics for verification