Probabilistically Tightened Linear Relaxation-based Perturbation Analysis for Neural Network Verification

2025-07-07

AI Summary

Neural network verification suffers from overly loose linear relaxations, leading to high conservatism and low computational efficiency. To address this, we propose a probabilistic tightening mechanism that integrates LiRPA-based formal bound analysis with stochastic sampling feedback to dynamically refine intermediate-layer reachable set estimates while preserving mathematical rigor. Our method constructs a lightweight probabilistic model via minimal sampling, enabling adaptive tightening of output bounds with negligible additional computational overhead. Evaluated on standard neural network verification benchmarks (e.g., VNN-COMP), it achieves an average 3.31× improvement in robustness certification rate over state-of-the-art tools. Notably, it successfully verifies challenging cases where leading approaches fail, demonstrating scalability and reliability for high-confidence (≥99%) verification. This work establishes a new paradigm for efficient, rigorous, and scalable neural network verification.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

We present $ extbf{P}$robabilistically $ extbf{T}$ightened $ extbf{Li}$near $ extbf{R}$elaxation-based $ extbf{P}$erturbation $ extbf{A}$nalysis ($ exttt{PT-LiRPA}$), a novel framework that combines over-approximation techniques from LiRPA-based approaches with a sampling-based method to compute tight intermediate reachable sets. In detail, we show that with negligible computational overhead, $ exttt{PT-LiRPA}$ exploiting the estimated reachable sets, significantly tightens the lower and upper linear bounds of a neural network's output, reducing the computational cost of formal verification tools while providing probabilistic guarantees on verification soundness. Extensive experiments on standard formal verification benchmarks, including the International Verification of Neural Networks Competition, show that our $ exttt{PT-LiRPA}$-based verifier improves robustness certificates by up to 3.31X and 2.26X compared to related work. Importantly, our probabilistic approach results in a valuable solution for challenging competition entries where state-of-the-art formal verification methods fail, allowing us to provide answers with high confidence (i.e., at least 99%).

Problem

• Combines over-approximation and sampling for tight neural network verification
• Reduces computational cost while ensuring probabilistic verification guarantees
• Improves robustness certificates significantly compared to existing methods

Innovation

• Combines over-approximation with sampling-based reachable sets
• Tightens neural network output bounds efficiently
• Provides probabilistic guarantees on verification soundness