Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms

2026-05-29

AI Summary

This work addresses a key limitation of conventional quantization methods, which optimize for minimal mean squared error yet often fail to preserve the inner products between vectors and arbitrary inputs, thereby degrading downstream task performance. The authors propose a novel quantization objective centered explicitly on inner product preservation, uncovering its intrinsic connection to Adaptive Stochastic Quantization (ASQ). Building on this insight, they design an unbiased, adaptive, and computationally efficient quantization algorithm that provably approximates inner product structures under both worst-case and average-case scenarios. The method offers strong theoretical guarantees while demonstrating practical efficacy across diverse data distributions. Empirically, their ASQ implementation achieves 2–10× speedups over current state-of-the-art approaches without sacrificing quantization accuracy.

Abstract

Quantization is a fundamental tool used to compress datasets, neural network weights, and memory usage in a range of computational tasks. Many downstream applications of vector quantization perform inner products with arbitrary inputs. This motivates the study of inner product aware quantization schemes that approximately preserve inner products with unseen vectors -- in contrast to simply minimizing the mean-squared error. In this work, we formulate objectives that capture natural desiderata and develop adaptive and unbiased quantization methods that approximately preserve inner products with worst-case and average-case inputs. An analysis of these objectives shows a tight connection with the well-studied notion of Adaptive Stochastic Quantization (ASQ). We develop provably fast exact and approximate algorithms for our objectives. Our theoretical results inspire efficient practical algorithms that perform well across a variety of workload distributions. They also lead to practical algorithms for standard ASQ which are 2-10$\times$ faster than prior state-of-the-art methods while maintaining quality. These theoretical and empirical results contribute towards making adaptive quantization techniques more efficient and tractable in practical settings.

Problem

• inner product
• quantization
• adaptive algorithms
• vector compression
• unseen vectors

Innovation

• inner product aware quantization
• adaptive stochastic quantization
• unbiased quantization
• fast quantization algorithms
• vector quantization