Verifying Neural Networks with Reinforcement Learning

2026-09-28

AI Summary

This study addresses the limitation of branching heuristics in neural network verification, which typically rely on static greedy decisions and lack long-term efficiency optimization. To overcome this, we propose RSB, a framework that introduces reinforcement learning to dynamically adjust baseline heuristic scores. Employing an Actor-Critic architecture combined with graph embeddings and attention mechanisms, RSB generates adaptive weights to reconstruct neuron branching guidance strategies, enabling forward-looking decisions through future reward prediction. Evaluated on 600 benchmark instances, RSB solves 11% more problems than existing state-of-the-art methods while reducing the number of explored branches by 50%. These results demonstrate significant improvements in both verification efficiency and coverage, establishing reinforcement learning as a promising paradigm for optimizing search strategies in formal neural network verification.

Abstract

Formal verification can play a key role in ensuring the reliability of Deep Neural Networks (DNNs) deployed in safety-critical systems. Modern DNN verifiers employ a branch-and-bound framework, which alternates between branching (splitting into smaller subproblems) and bounding (pruning subproblems) to efficiently explore the verification space. However, existing branching heuristics make greedy decisions based on static scoring functions. They do not anticipate long-term efficiency or leverage the growing availability of verification data to improve performance. This work introduces RSB, a reinforcement learning framework that learns to refine baseline branching heuristics. It trains an actor-critic architecture to maximize cumulative future rewards rather than immediate scores. The actor generates attention weights from observations of raw neuron features and learned graph embeddings, which rescale baseline heuristic scores to guide neuron branching. Evaluation on 600 challenging instances demonstrates that RSB consistently outperforms state-of-the-art branching heuristics, solving 11% more instances while reducing branch exploration by 50%.

Problem

• Neural Network Verification
• Formal Verification
• Branch-and-Bound
• Branching Heuristics

Innovation

• Reinforcement Learning
• Formal Verification
• Branch-and-Bound
• Actor-Critic
• Graph Embeddings