🤖 AI Summary
Neural network quantization often suffers from significant performance degradation due to parameter discretization. This work proposes a novel quantization paradigm that circumvents the need for straight-through estimators or explicit discretization during training. By exploring low-loss subspaces in the weight space and optimizing a quantization-aware linear path, the method steers quantized models naturally toward high-accuracy regions. Remarkably, directly quantizing only the interpolated points within this subspace achieves accuracy comparable to quantization-aware training and substantially outperforms conventional post-training quantization approaches. The study further highlights the critical role of low-loss subspace structures in enabling efficient and accurate quantization.
📝 Abstract
Neural network quantization aims to find a discrete representation of parameters that preserves the performance of a full-precision (FP) model as faithfully as possible. Enforcing discrete constraints perturbs parameters away from a well-optimized minimum, generally resulting in performance degradation. Recent studies indicate that low-loss FP solutions are not isolated, but instead belong to connected low-loss subspaces of the loss landscape, where the loss maintains nearly the same minimum value. Models sampled from these subspaces are diverse and retain high accuracy. This raises the question: can a quantized model be constructed to lie within a low-loss subspace of the FP model, thereby automatically preserving performance? We address this question by learning quantization-aware linear paths in weight space optimized to minimize loss. We demonstrate that the midpoint of the resulting subspace is, by design, quantization-friendly and that its direct quantization yields performance comparable to that of quantization-aware training. The proposed procedure offers a novel perspective on weight quantization and, in contrast to conventional methods, neither relies on the straight-through estimator nor involves explicit discretization during training.