A Hand-Checkable Proof That Two Hidden ReLU Layers Compute the Maximum of Six Numbers

2026-10-03

AI Summary

This study addresses the long-standing reliance on computer-assisted verification for depth lower bounds of ReLU networks computing the maximum function, which has lacked human-verifiable analytical proofs. Focusing on the six-input case, this work employs computational search to discover an explicit hexagonal identity and its local structure. By integrating algebraic cancellation techniques, it establishes a self-contained analytical proof framework and provides explicit network constructions with zero biases and rational weights, achieving hidden layer widths of 17 and 41. The primary contribution lies in rigorously establishing the depth bound for two-hidden-layer ReLU networks computing the maximum of six numbers. This represents the first fully human-verifiable mathematical proof that requires no program execution, thereby overcoming the limitations of traditional computational verification approaches.

Abstract

Exactly computing the maximum function is a standard test case for studying depth in ReLU networks. Two hidden layers are known to suffice for up to twelve inputs through computer-assisted constructions. For six real inputs, we give an explicit hexagon identity whose local structure yields a self-contained analytical proof of this depth bound. The identity was found by computer-assisted search; we prove it through explicit cancellations that can be checked entirely by hand, without executing a verification program. The identity also yields an explicit network with hidden widths $17$ and $41$, zero biases, and rational weights.

Problem

• ReLU networks
• maximum function
• network depth
• hexagon identity

Innovation

• ReLU networks
• maximum function
• depth bounds
• hexagon identity
• computer-assisted proof