Adaptive Power Iteration Method for Differentially Private PCA

2026-02-12

AI Summary

This work addresses the problem of efficiently approximating the leading principal components of high-dimensional data matrices under $(\varepsilon, \delta)$-differential privacy, where neighboring datasets differ by a single row. We propose a noise-filtering mechanism that adapts to the matrix coherence and is integrated into the power iteration algorithm, significantly improving the accuracy of principal component estimation while preserving privacy. By moving beyond worst-case analyses, our approach achieves particularly strong performance on low-coherence matrices. Furthermore, this method extends the framework of Hardt and Roth from the more restrictive user-level privacy model to the more practical and widely applicable row-level privacy setting.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We study $(\epsilon,\delta)$-differentially private algorithms for the problem of approximately computing the top singular vector of a matrix $A\in\mathbb{R}^{n\times d}$ where each row of $A$ is a datapoint in $\mathbb{R}^{d}$. In our privacy model, neighboring inputs differ by one single row/datapoint. We study the private variant of the power iteration method, which is widely adopted in practice. Our algorithm is based on a filtering technique which adapts to the coherence parameter of the input matrix. This technique provides a utility that goes beyond the worst-case guarantees for matrices with low coherence parameter. Our work departs from and complements the work by Hardt-Roth (STOC 2013) which designed a private power iteration method for the privacy model where neighboring inputs differ in one single entry by at most 1.

Problem

• differentially private PCA
• power iteration
• singular vector
• coherence parameter
• privacy

Innovation

• differentially private PCA
• adaptive power iteration
• coherence parameter
• row-level privacy
• singular vector computation