When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation

2026-07-25

AI Summary

This work investigates whether the precision loss incurred by low-bit residual computation can be compensated through increased network depth under a fixed input–output mapping. The quantized residual system is modeled as a scheduling problem selecting operations from a low-bit library, with its infinite-depth limit characterized via relaxed control theory. The study analyzes how execution semantics and routing mechanisms affect the capacity for depth-based compensation. Key contributions include establishing a structural accuracy lower bound that delineates the theoretical limits of depth compensation, revealing that incremental error feedback circumvents the depth penalty induced by state write-back and leads to a coherent conservation law, and introducing a pre-training verifiable feasibility criterion based on dual bounds. Theoretically, pure scheduling converges to the relaxed solution at rates of either $O(D^{-1})$ or $O(D^{-\phi}+D^{-1})$, requiring depth $D=\Theta(L)$ to match high-precision performance. Core arguments are formally verified in Lean 4, accompanied by a complete software implementation.

Abstract

When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map? We model a quantized residual system over a fixed horizon as a pure schedule selecting fields from a declared low-bit operation library, and use relaxed controls to characterize its infinite-depth limit. The distance from the target to the closed relaxed reachable set is the exact structural floor: no increase in depth can remove it for that library. Pure schedules approach the relaxed class at rate $O(D^{-1})$ under bounded-variation time dependence and $O(D^{-\vartheta}+D^{-1})$ under Holder dependence of exponent $\vartheta$. Execution arithmetic can reverse this conclusion: full-state write-back introduces a $Dρ_z$ penalty and can freeze residual updates, whereas increment error feedback replaces this growth by a bounded carry term and obeys an exact common-lattice conservation law. A fixed-teacher converse makes this rate sharp: for coherent depth-$L$ first-order high-precision comparators, accuracy matching requires $D=Θ(L)$. Learned codebooks add a metadata resource, while state-dependent routing introduces hybrid event conditions. Verified primal and dual bounds yield feasible, impossible, or unresolved decisions before training. Companion software implements the workflow, and Lean 4 machine-checks the exact discrete core. Depth replaces precision only relative to a declared library, horizon, execution semantics, and routing model.

Problem

• quantized neural computation
• depth-precision tradeoff
• residual networks
• numerical precision
• low-bit arithmetic

Innovation

• quantized neural computation
• relaxed controls
• residual depth
• error feedback
• resource theory