SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant

2026-08-05

AI Summary

This work addresses the challenge in distributed optimization where existing vector quantization methods suffer from high variance due to dimensionality dependence, making it difficult to simultaneously achieve local differential privacy and low communication overhead. To overcome this, the authors propose the Subsampled Stochastic TurboQuant (SSTQ) framework, which integrates overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization—realized through two variants: Flat Randomized Response and Metric-Aware Laplace. By designing a privacy-aware codebook objective, SSTQ reduces the codebook-dependent mean squared error (MSE) scaling from O(4ᵇ) to O(2ᵇ), achieving optimal MSE scaling with only ⌈log₂N⌉ + b bits per client. Experiments on CIFAR-10 and Fashion-MNIST federated learning tasks demonstrate that SSTQ significantly outperforms baseline methods, offering a superior trade-off between utility and communication efficiency.

Abstract

Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD, use high-dimensional geometric constructions but incur unfavorable dimension-dependent variance. In this work, we propose Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization. SSTQ includes two variants: a Flat Randomized Response version and a Metric-Aware Laplace version, the latter being better suited to higher codebook bit-width regimes. We show that SSTQ achieves optimal mean squared error scaling while using only $\lceil \log_2 N \rceil + b$ bits per client, where $N = Θ(d)$ is the frame size. We also derive a surrogate privacy-aware codebook objective that reduces the codebook-dependent MSE scaling from $O(4^b)$ to $O(2^b)$. Finally, we empirically evaluate SSTQ against established baselines on federated learning tasks using CIFAR-10 and Fashion-MNIST, demonstrating favorable utility and communication efficiency.

Problem

• local differential privacy
• vector quantization
• communication efficiency
• distributed optimization
• privacy-preserving

Innovation

• Vector Quantization
• Local Differential Privacy
• Federated Learning
• Communication Efficiency
• Tight Frames