The Fast Mixing Mechanism for Differential Privacy

2026-05-28

AI Summary

This work addresses the computational inefficiency of existing Gaussian sketching methods in differentially private linear regression, despite their strong utility guarantees. The authors propose a novel differentially private sketching mechanism based on fast structured transforms—such as the Hadamard transform—to enable efficient data compression in ordinary least squares estimation. Their approach is the first to simultaneously achieve near-optimal accuracy comparable to Gaussian sketches and computational speed approaching that of non-private fast sketching algorithms, all under rigorous differential privacy guarantees. Under typical parameter settings, the method significantly outperforms current differentially private alternatives in runtime while delivering state-of-the-art trade-offs between efficiency and statistical utility.

Abstract

Randomized sketching is a central tool for compressing large-scale optimization problems while preserving accuracy. In particular, sketches that are based on structured matrices, such as the Hadamard matrix, can be applied efficiently and often yield solutions that approximate those of the original problem at much lower computational cost. In differential privacy (DP), Gaussian sketching has been used to solve DP linear regression, beginning with \citet{sheffet2017differentially, sheffet2019old} and later refined by \citet{lev2025gaussianmix, lev2026near}. However, although these methods achieve strong utility guarantees, they usually do not improve runtime over classical DP approaches. In this work, we introduce a new DP sketching mechanism based on fast transforms, which, in certain cases, matches the runtime of classical fast sketching methods. We prove state-of-the-art privacy guarantees for this mechanism and show that, in favorable regimes, they match those of the Gaussian sketch up to a constant factor. As an application, we combine this mechanism with recent sketch-based methods for DP linear regression to obtain a new algorithm with strong utility and improved runtime. We establish privacy and accuracy guarantees for this algorithm, yielding, to the best of our knowledge, the first fast method for DP ordinary least squares.

Problem

• differential privacy
• randomized sketching
• fast transforms
• linear regression
• computational efficiency

Innovation

• fast sketching
• differential privacy
• structured random matrices
• linear regression
• privacy-preserving optimization