Random Parameter Noise Does Not Make Exact ReLU Verification Easy

2026-07-15

AI Summary

This work investigates the computational complexity of exactly verifying the maximum output of ReLU neural networks under parameter perturbations involving Gaussian noise, clipping, and exact rounding to a dyadic rational grid. By integrating techniques from exact gap embedding, quantized robustness analysis, Gaussian concentration inequalities, and complexity-theoretic reductions, the paper establishes—for the first time—that no expected polynomial-time complete and sound verification algorithm exists even for small, non-degenerate single-hidden-layer ReLU networks, provided the noise level is fixed (e.g., σ★ = 2⁻¹¹). The result holds with probability at least 1 − e⁻ᵐ⁄⁸, preserving a linear verification gap tied to E3SAT, thereby refuting the common assumption that random noise simplifies verification and demonstrating the problem’s computational hardness in the BPP sense.

Abstract

We study exact verification of ReLU networks in an adversarial smoothed model. Every network weight and bias is independently perturbed by Gaussian noise, clipped to $[-2,2]$, and rounded to the exact dyadic grid determined by the input bit complexity. We show that, under the standard assumption $\mathrm{NP}\not\subseteq\mathrm{BPP}$, there is no sound and complete verifier whose expected running time is polynomial in network size, bit complexity, and inverse noise level for every base instance. The conclusion already holds at the fixed noise level $σ_\star=2^{-11}$ for one-hidden-layer networks over a unit box, with hidden fan-in at most three and base coefficients in $[-1,1]$. The proof combines an exact gap embedding with a quantitative robustness argument. For every E3SAT formula $Φ$ with $m$ clauses, a four-ReLU-per-clause construction satisfies $\max_{x\in[0,1]^n} g_Φ(x)=(m-\operatorname{unsat}(Φ))/3$, and coordinatewise threshold rounding never decreases the objective. A weighted parameter-sensitivity inequality and Gaussian concentration then show that a verification gap linear in $m$ survives the aggregate perturbation of all coefficients with probability at least $1-e^{-m/8}$. The proof includes clipping, exact dyadic rounding, output-layer perturbations, polynomial-bit sampling of the rounded Gaussian law, and the conversion from expected smoothed running time to a BPP algorithm. Computational checks test the exact identity and illustrate the different scaling of extensive and constant gaps; they are diagnostics rather than evidence for the complexity theorem. The result concerns worst-case base networks in the stated absolute-noise model, but it shows that parameter nondegeneracy alone does not yield a universal smoothed-polynomial guarantee for exact verification.

Problem

• ReLU verification
• smoothed analysis
• random parameter noise
• computational complexity
• adversarial robustness

Innovation

• smoothed analysis
• exact verification
• ReLU networks
• computational hardness
• parameter perturbation