quantization-aware training

Neural network quantization often suffers from significant performance degradation due to parameter discretization. This work proposes a novel quantization paradigm that circumvents the need for straight-through estimators or explicit discretization during training. By exploring low-loss subspaces in the weight space and optimizing a quantization-aware linear path, the method steers quantized models naturally toward high-accuracy regions. Remarkably, directly quantizing only the interpolated points within this subspace achieves accuracy comparable to quantization-aware training and substantially outperforms conventional post-training quantization approaches. The study further highlights the critical role of low-loss subspace structures in enabling efficient and accurate quantization.

To address the train-inference inconsistency caused by non-differentiable quantization in learned image compression, this paper proposes a two-stage training paradigm: first, end-to-end pretraining using standard differentiable quantization approximations; second, fine-tuning the decoder and entropy model under strict entropy-constrained quantization (e.g., Trellis-Coded Quantization), with supervision directly from the quantized latent variables. This work is the first to embed exact entropy-constrained quantization into the fine-tuning stage, eliminating modeling biases inherent in noise injection or rounding-based approximations and effectively bridging the gap between quantization distortion and rate-distortion optimization. Experiments on Kodak and TecNick demonstrate average BD-rate reductions of 1.0%–2.0%, up to 2.2%, with zero increase in inference complexity.

To address deployment constraints on resource-limited devices, this paper proposes a fine-tuning-free post-training quantization (PTQ) method for non-uniform weight quantization. To overcome the accuracy bottlenecks of conventional uniform or channel-wise quantization, we introduce, for the first time in PTQ, a theoretically grounded noise-minimization mechanism that jointly optimizes clipping thresholds and scaling factors. Our approach leverages gradient-free statistical modeling, hierarchical non-uniform clustering, and convex optimization to achieve distribution-adaptive quantization. Evaluated on real-world datasets, our method achieves 4–8× model compression, 3.2× inference speedup, and less than 0.3% Top-1 accuracy degradation—substantially outperforming state-of-the-art PTQ baselines. Key contributions include: (i) a theoretically guaranteed optimal non-uniform quantization design; (ii) the first provably noise-minimizing PTQ framework; and (iii) a highly efficient, entirely fine-tuning-free implementation.

To address the accuracy degradation of large language models (LLMs) under low-bit quantization, this paper proposes a post-training model expansion method that dynamically enhances 4-bit quantized LLM performance without retraining. The core innovation lies in the first systematic validation of synergistic co-design between post-training expansion and quantization. We introduce a selective, progressive parameter expansion mechanism integrated with Hadamard rotation, high-precision retention of sensitive weights, layer-wise expansion, and quantization-aware structural adaptation. Evaluated on Llama3-1B, our approach achieves full 4-bit weight and activation quantization, attaining an average zero-shot accuracy improvement of 3% over QuaRot and SpinQuant, with only a 5% parameter increase. The resulting model size is reduced by 3.8% relative to the BF16 baseline—breaking the conventional “compression implies parameter reduction” paradigm and enabling joint optimization of accuracy and efficiency.

Extremely low-bit (e.g., 4-bit) post-training quantization (PTQ) suffers severe accuracy degradation due to quantization noise. To address this, we propose the first model-level series expansion framework for PTQ—requiring neither calibration data nor fine-tuning—that losslessly decomposes a floating-point model into multiple low-bit basis models. Our key contributions include: (i) the first application of series expansion to neural network quantization; (ii) a multi-granularity (tensor-, layer-, and model-level) low-bit basis expansion with rigorous convergence guarantees; and (iii) the design of AbelianAdd/Mul operators that form an Abelian group, ensuring parallelizability and commutativity. Experiments demonstrate state-of-the-art performance: 4-bit ResNet-50 achieves 77.03% Top-1 accuracy—surpassing its full-precision baseline—and establishes new SOTA across diverse architectures and tasks in low-bit PTQ.

To address efficiency and accuracy degradation in low-precision training caused by post-hoc quantization of continuous parameter updates, this paper proposes a novel **discrete update paradigm**: directly designing update rules operating exclusively in the discrete parameter space, thereby eliminating real-valued computation and storage. We establish, for the first time, a convergence theory for discrete update schemes and construct concrete polynomial-based mechanisms—e.g., multinomial updates. Grounded in discrete dynamical systems modeling, our approach fully removes dependence on continuous variables and naturally aligns with intrinsically discrete model architectures. Experiments demonstrate that, while preserving model accuracy, the method significantly improves training efficiency and hardware compatibility. It provides a theoretically rigorous yet implementationally simple framework for ultra-low-bit (e.g., sub-2-bit) neural network training.

This work addresses the significant performance degradation often observed in low-bit (e.g., 2/4-bit) post-training quantization (PTQ), which stems from the sensitivity of full-precision models to quantization error. To mitigate this issue, the authors propose Efficient Tuning Before Quantization (ETBQ), a lightweight pre-tuning method that injects simulated quantization noise during full-precision training. This guides optimization toward loss landscapes that are inherently robust to quantization, thereby enhancing subsequent PTQ performance. Notably, ETBQ avoids explicit fake quantization operations and yields full-precision models compatible with any PTQ backend without modification. Experimental results demonstrate consistent improvements across benchmarks: under W2A4 settings, it achieves a 2.14% gain in top-1 accuracy on Tiny-ImageNet and a 5.80% increase in mIoU on Cityscapes, validating its effectiveness and broad applicability.

This work addresses the performance limitations of quantization-aware training (QAT) in ultra-low-bit (2–4 bit) settings, where gradient mismatch and training instability often hinder convergence. To overcome these challenges, the authors propose StableQAT, which introduces discrete Fourier analysis into QAT for the first time to construct a family of lightweight, smooth, and bounded gradient surrogate functions that rigorously generalize the straight-through estimator (STE). This approach significantly enhances training stability, model accuracy, and robustness to hyperparameter choices, all while incurring negligible additional training overhead. As a result, StableQAT enables efficient and reliable ultra-low-bit quantization-aware training without compromising computational efficiency.

This work addresses the challenge of balancing accuracy and generalization in neural network quantization by proposing a soft quantization method during training. The approach introduces short-range attractive coupling among weights to encourage automatic discretization of the weight distribution, enabling mixed-precision compression without complex scheduling schemes. Relying on only two hyperparameters, the method offers both simplicity and flexibility, providing a novel tool for studying the trade-off between model compression and generalization. Experimental results on ResNet-20 with CIFAR-10 demonstrate that the proposed technique outperforms post-training quantization methods based on histogram equalization, achieving higher accuracy in compressed models.

Straight-through estimators (STE) are widely adopted in quantized neural network training due to the discrete, non-differentiable nature of the objective, yet the mechanistic role of hyperparameters—such as bit-width and quantization range—in governing STE’s learning dynamics remains poorly understood. Method: We rigorously derive, in the high-dimensional limit, that STE dynamics converge to a deterministic differential equation. We analyze fixed points to quantify the asymptotic bias relative to unquantized linear models and extend the framework to nonlinear settings—incorporating non-convex optimization and SGD theory—to model proxy-gradient dynamics for both weights and inputs. Results: Our analysis reveals that quantization range primarily governs the duration of the generalization error plateau, while bit-width controls convergence speed. We observe and explain the characteristic plateau-and-drop behavior in generalization error evolution. This work establishes the first analytically tractable, high-dimensional dynamical systems framework for efficient quantized training.