The Ball and the Box: Two Geometries of Computation in Superposition

2026-10-08

AI Summary

This study addresses the unresolved theoretical characterization of the dimensionality required to compute Boolean gates under neural representation superposition. By leveraging Gaussian random dictionaries and sparse Boolean inputs, combined with threshold-layer analysis and combinatorial optimization techniques, this work reveals that expected error and joint reliability correspond respectively to “ball” and “box” geometric structures, which are subsequently exploited to optimize shared readout weights. The primary contributions include deriving explicit dimensional thresholds for conjunction, disjunction, and majority gates, elucidating the mechanisms by which shared readouts induce performance gaps, and characterizing near-critical transition behaviors. Furthermore, the proposed theoretical predictions demonstrate strong agreement with exact numerical simulations, providing a rigorous analytical framework for understanding computational capacity limits in superposed neural representations.

Abstract

Neural representations can encode more features than they have dimensions, a phenomenon known as superposition. We study the dimension needed to compute Boolean gates from such representations. For a single threshold layer with a Gaussian random dictionary and uniformly random sparse Boolean inputs, we derive sharp dimension thresholds under two error criteria. A vanishing expected error count can require more dimensions than correctness of every output with high probability. Shared reads explain the gap: rare realizations can produce many errors at once. The expected-count threshold has ball geometry, while joint reliability has box geometry when a gate is evaluated on every feature tuple. Optimizing shared readout weights and biases gives explicit thresholds for conjunction, disjunction, and majority. For pairwise conjunction, the analysis also describes the transition near the threshold, in agreement with exact simulations.

Problem

• superposition
• Boolean gates
• dimension threshold
• neural representations

Innovation

• superposition
• Boolean gates
• dimension thresholds
• shared reads
• computational geometry