Quadratic Characterizations for Reachability Analysis of Neural Networks

2026-05-19

AI Summary

Existing quadratic constraint approaches for characterizing neural network activation functions are overly conservative, limiting the precision of reachability and safety analyses. This work proposes a domain-dependent framework for verifiable quadratic inequalities: it generates candidate constraints via local sampling and employs sum-of-squares (SOS) certificates to ensure global validity, yielding tight and sound quadratic representations for scalar nonlinearities. The method transcends the limitations of conventional sector or slope bounds by incorporating neuron-wise dependencies and local bound refinement—particularly for ReLU networks—to reduce conservatism. It is compatible with convex quadratic programming, semialgebraic set descriptions, and integral quadratic constraint (IQC) techniques. Experiments demonstrate that the framework significantly improves analysis accuracy for smooth activations such as tanh and extends effectively to systems involving saturation-type nonlinearities.

Abstract

Quadratic constraints (QCs) are widely used to characterize nonlinearities and uncertainties, but generic analytical characterizations can be conservative on bounded domains. This paper develops a framework for constructing verified quadratic characterizations of scalar relations in the two-dimensional real plane. Candidate quadratic inequalities are locally generated by solving convex quadratic programs using samples from the relation and exterior sample points. They are then verified globally using sum-of-squares certificates over an exact semialgebraic description or, in the case of nonpolynomial relations, over relaxed polynomial descriptions. The resulting verified constraints define a sound overapproximation of the scalar relations over the considered domains. These constraints are directly compatible with existing analysis frameworks based on QCs and pointwise integral quadratic constraints (IQCs) for static nonlinearities and uncertainties, and they can also be embedded in QC-based semidefinite programs for reachability and safety analysis of feedforward neural networks. For smooth activations such as $\tanh$, the method yields domain-dependent quadratic characterizations that constitute an alternative to generic sector- or slope-based descriptions. For ReLU networks, we give methods to reduce conservatism in QC-based reachability analysis of feedforward networks by exploiting dependencies between neurons and tighter local bounds. Numerical examples demonstrate improved reachability results for smooth activations, reduced conservatism for ReLU networks, and applicability beyond neural networks through an example involving saturation.

Problem

• quadratic constraints
• reachability analysis
• neural networks
• nonlinearities
• conservatism

Innovation

• quadratic constraints
• sum-of-squares verification
• reachability analysis
• neural network verification
• domain-dependent characterization