Rejection-Sampled Universal Quantization for Smaller Quantization Errors

2024-02-05

International Symposium on Information Theory

AI Summary

Conventional lattice-based vector quantizers suffer from error distributions constrained by the geometry of fundamental Voronoi cells, limiting flexibility and performance in high-dimensional settings. Method: We propose a general stochastic vector quantizer based on rejection sampling that explicitly reshapes the quantization error distribution to be uniform within a Euclidean ball—achieving, for the first time, strict input-independent uniformity over the ball. Contributions/Results: Theoretical analysis shows that, at fixed entropy, our quantizer achieves lower maximum error than optimal lattice quantizers in dimensions 5–48, and lower mean squared error in 35–48 dimensions. Furthermore, for additive noise channels satisfying mild conditions (e.g., AWGN), we characterize the high-SNR channel capacity limit under single-channel simulation, with an error bound of ±1.45 bits. This work overcomes the geometric bottleneck inherent in lattice quantization and establishes a new paradigm for high-resolution quantization and channel simulation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

We construct a randomized vector quantizer which has a smaller maximum error compared to all known lattice quantizers with the same entropy for dimensions 5, 6,…, 48, and also has a smaller mean squared error compared to known lattice quantizers with the same entropy for dimensions 35,…, 48, in the high resolution limit. Moreover, our randomized quantizer has a desirable property that the quantization error is always uniform over the ball and independent of the input. Our construction is based on applying rejection sampling on universal quantization, which allows us to shape the error distribution to be any continuous distribution, not only uniform distributions over basic cells of a lattice as in conventional dithered quantization. We also characterize the high SNR limit of one-shot channel simulation for any additive noise channel under a mild assumption (e.g., the AWGN channel), up to an additive constant of 1.45 bits.

Problem

• Constructing a randomized vector quantizer with smaller maximum error
• Achieving uniform and input-independent quantization error distribution
• Characterizing high SNR limit for one-shot channel simulation

Innovation

• Rejection sampling shapes universal quantization error distribution
• Randomized quantizer achieves smaller maximum and mean squared errors
• Method enables any continuous distribution for quantization error