Fractal Gadgets for Neural Networks: The Complexity of the Narrow Regime

2026-10-05

AI Summary

This study addresses the theoretical gap regarding the computational complexity of verifying narrow deep ReLU neural networks under width constraints. It proposes a preprocessing construction based on width-2 fractal Cantor subnetworks, which leverages ReLU activation functions to formulate fractal iterative mappings. This approach enables a lossless reduction from continuous-domain verification to discrete-point verification without increasing network width. The work rigorously establishes that this verification problem is NP-complete at width 4, while further delineating the NP-completeness for widths 3 and 4 alongside complexity boundaries for multiple variants. Collectively, these results systematically reveal the intrinsic computational complexity inherent in narrow neural networks.

Abstract

We study the verification problem for deep narrow ReLU neural networks: given a network of bounded width computing a piecewise-affine map on [0,1], does some input satisfy a prescribed output constraint? Classical NP-hardness proofs for ReLU verification use one neuron per Boolean variable and say nothing about networks of small constant width, while width-1 networks are easy to verify. We show that verification of ReLU networks is NP-complete at width 4 for arbitrary inputs in [0,1]. When inputs are restricted to a natural discrete encoding set, NP-completeness already holds at width 3. Together with polynomial-time decidability at width 1, this leaves open only width 2 on the encoding set, and widths 2 and 3 on [0,1]. The technical core is a fractal preprocessing gadget: a width-2 ReLU subnetwork whose iterate vanishes precisely near a finite Cantor-like subset of [0,1] with 2^n points. It reduces verification of a continuous function on [0,1] to verification on 2^n discrete points without increasing the width, and is the missing ingredient for width-bounded hardness reductions. The same construction yields further results at width 3 on the encoding set: the universal problem is coNP-complete, counting zeros is #P-complete, a majority variant is PP-complete, and approximating the minimum output within a constant gap inherited from Max-3Sat is NP-hard. The NP, coNP and inapproximability results lift to all of [0,1] at width 4; lifting counting and majority, and lifting at width 3, remain open.

Problem

• Neural network verification
• ReLU networks
• Computational complexity
• Narrow networks
• NP-completeness

Innovation

• Neural network verification
• Fractal gadget
• Narrow ReLU networks
• Computational complexity
• NP-completeness