🤖 AI Summary
This study challenges the common assumption that convergence in FP32 implies suitability for quantization by revealing a sharp degradation in INT4 performance after FP32 convergence. Analyzing all 154 training checkpoints of Pythia-160M using calibration-free per-group INT4 probing, the authors identify a three-phase explosive growth in quantization error post-convergence and, for the first time, link the onset of this collapse to FP32 perplexity convergence rather than learning rate decay. They propose Oscillatory Lock-In, a novel learning rate schedule combined with kurtosis-based outlier filtering, which significantly enhances INT4 robustness. In multi-schedule comparisons, this approach effectively mitigates the surge in late-stage quantization error—from 11% to 517%—reducing it on average by 2.2 percentage points (p<0.0001), thereby validating the critical role of schedule amplitude calibration.
📝 Abstract
Post-training quantization (PTQ) assumes that a well-converged model is a quantization-ready model. We show this assumption fails in a structured, measurable, and previously uncharacterized way. Using a calibration-free per-group INT4 probe applied to all 154 publicly available Pythia-160m training checkpoints, we identify a three-phase divergence structure: a rapid-learning phase where both FP32 perplexity and quantization robustness improve together, a meta-stable plateau lasting roughly 70,000 steps where FP32 perplexity stagnates but INT4 gap remains bounded, and an explosive divergence phase where the INT4 gap compounds from 11% to 517% while FP32 perplexity barely moves. Critically, this divergence begins not when the learning rate starts decaying, but precisely when FP32 perplexity converges a finer-grained onset predictor that implies post-convergence weight updates, rather than decay magnitude alone, are the proximate cause. We further show that INT8 quantization is entirely immune throughout all three phases, constraining the mechanism to the coarseness of the 16-level INT4 grid specifically, and rule out weight outlier accumulation as the mechanism via direct kurtosis measurement. Finally, we conduct a controlled fork experiment from the pre-divergence checkpoint comparing three learning rate schedules (cosine continuation, SGDR warm restarts, and our proposed Oscillatory Lock-In) across nine independent runs. SGDR uniformly accelerates divergence (0/9 pairwise wins against cosine), while OLI's settled cool phases reduce the INT4 gap by 2.2 percentage points on average (t = -5.46, p < 0.0001), demonstrating that schedule amplitude calibration, not oscillation alone, determines whether perturbation helps or hurts. Our code, probe implementation, and all 154-checkpoint audit results are released publicly.