Pairwise-Independent Dithering for Single-Stage Hadamard Quantization

2026-08-03

AI Summary

This work addresses the high communication overhead and large constant factors in error bounds associated with high-dimensional vector quantization by proposing a single-stage Hadamard quantization method based on pairwise independent dithering. By applying pairwise independent dithering in the Hadamard basis, the method achieves unbiased inner product estimation and, for the first time, demonstrates that a single-stage architecture can attain theoretically optimal performance. This eliminates the need for residual quantization in conventional two-stage approaches, removing the O(d)-bit overhead and reducing the error bound constant by approximately 5.93×. The resulting inner product estimation error bound is dimension-independent: as the number of bits b → ∞, the expected error is at most (3π√3/2 + o(1))·‖y‖₂²/(d·4ᵇ), significantly outperforming existing methods.

Abstract

Quantizing high-dimensional vectors is fundamental to similarity search, distributed learning, and model compression. Feng, Indyk, Kapralov, Krachun, and Prokhorov established sharp guarantees for an unbiased dithered quantizer based on a randomized Hadamard transform [FIK+26]. Their $1/d$-scale inner-product estimator, however, uses a second randomized transform and residual quantization, increasing both communication and the leading constant in the proved bound. We show that this extra stage is unnecessary: pairwise-independent dithers across Hadamard coordinates suffice. The resulting unbiased single-stage estimator uses $b$ bits per coordinate and achieves \[ \mathbb{E}\!\left[ \left|\left\langle y,\widehat{x}-x\right\rangle\right|^2 \right] \leq \left(\frac{3π\sqrt{3}}{2}+o(1)\right) \frac{\lVert y\rVert_2^2}{d\,4^b}, \] as $b\to\infty$, with a dimension-free $o(1)$ term uniform over unit inputs and fixed queries. Compared with the two-stage construction of Feng et al., it eliminates the residual-stage $O(d)$-bit payload and reduces the leading upper-bound constant by a factor of approximately $5.93$. The proof was first obtained using a fully automated Gemini-based agentic system developed internally at Google. The authors have verified the proof and edited it for clarity of presentation.

Problem

• Hadamard quantization
• dithering
• similarity search
• unbiased estimation
• single-stage quantization

Innovation

• pairwise-independent dithering
• single-stage quantization
• Hadamard transform
• unbiased estimator
• dimension-free bound