ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

2026-08-07

AI Summary

This work addresses a key limitation in existing post-training quantization (PTQ) methods, which typically treat quantized integer weights as final and lack mechanisms for further refinement. The authors propose a novel backpropagation-free, fixed-grid discrete optimization approach that iteratively redistributes the weights of an already quantized model to minimize mean squared reconstruction error, all while preserving the original quantization format. By explicitly treating the quantized model as an optimizable discrete solution, this method introduces an initialization-agnostic, plug-and-play post-processing stage that overcomes the one-shot nature of conventional PTQ. Extensive experiments demonstrate consistent performance gains across diverse model architectures, bit-widths, and downstream tasks, with particularly pronounced improvements under low-bit settings and simple PTQ initializations—often approaching or even surpassing the accuracy of GPTQ.

Abstract

Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models. Existing PTQ methods typically obtain an initial quantized model through heuristic rules or greedy optimization, and once quantization is completed the resulting integer assignments are usually treated as final. This observation motivates a complementary optimization stage within PTQ that keeps quantized weights improvable after an executable quantized model has been produced, while preserving the quantized format. We introduce ReQuant, a backpropagation-free fixed-grid refinement procedure for this stage. Agnostic to the PTQ initializer, ReQuant takes an existing quantized model as a feasible starting point and iteratively revisits its discrete weight assignments on the fixed quantization grid. Accepted updates strictly reduce the mean squared reconstruction error and remain on the original grid. In this way, ReQuant turns the initially fixed PTQ output into an iteratively optimizable discrete solution and serves as a plug-and-play post-processing stage for existing PTQ pipelines. Experiments across diverse model families, bit-widths, and downstream tasks show that ReQuant consistently improves quantized models from heterogeneous PTQ initializers, with especially large gains on simple initializers and lower bit-widths. Notably, ReQuant can refine a simple round-to-nearest initialization across multiple sweeps until it approaches or surpasses GPTAQ under the same quantization format. These results establish ReQuant as a practical complementary stage for further improving existing PTQ pipelines.

Problem

• post-training quantization
• quantized model refinement
• discrete optimization
• fixed-grid quantization
• model compression

Innovation

• post-training quantization
• discrete refinement
• fixed-grid optimization
• quantization grid
• plug-and-play