PREAMBLE: Private and Efficient Aggregation of Block Sparse Vectors and Applications

2025-03-14

AI Summary

This work addresses secure aggregation of high-dimensional sparse gradients in a two-server architecture. Methodologically, it introduces the first efficient privacy-preserving aggregation scheme tailored to block-sparse vectors—where non-zero entries form locally contiguous clusters. The approach extends distributed point functions (DPFs) to accommodate block-sparse structures and integrates block-sparse encoding, randomized sampling, and privacy amplification. Crucially, both communication and computational complexity are independent of vector dimensionality, without compromising security. Compared to baselines such as Prio, the scheme achieves substantial communication savings, negligible increase in noise variance, and near-optimal privacy budget utilization—approaching the theoretical lower bound of the Gaussian mechanism. It enables efficient federated learning aggregation for million-dimensional block-sparse gradients, achieving asymptotically optimal privacy-utility trade-offs.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Distributed Search
• Multiagent Systems: Mechanism Design

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

We revisit the problem of secure aggregation of high-dimensional vectors in a two-server system such as Prio. These systems are typically used to aggregate vectors such as gradients in private federated learning, where the aggregate itself is protected via noise addition to ensure differential privacy. Existing approaches require communication scaling with the dimensionality, and thus limit the dimensionality of vectors one can efficiently process in this setup. We propose PREAMBLE: Private Efficient Aggregation Mechanism for BLock-sparse Euclidean Vectors. PREAMBLE is a novel extension of distributed point functions that enables communication- and computation-efficient aggregation of block-sparse vectors, which are sparse vectors where the non-zero entries occur in a small number of clusters of consecutive coordinates. We then show that PREAMBLE can be combined with random sampling and privacy amplification by sampling results, to allow asymptotically optimal privacy-utility trade-offs for vector aggregation, at a fraction of the communication cost. When coupled with recent advances in numerical privacy accounting, our approach incurs a negligible overhead in noise variance, compared to the Gaussian mechanism used with Prio.

Problem

• Secure aggregation of high-dimensional vectors in two-server systems
• Efficient processing of block-sparse vectors with reduced communication
• Achieving optimal privacy-utility trade-offs in vector aggregation

Innovation

• Efficient aggregation of block-sparse vectors
• Combines random sampling with privacy amplification
• Reduces communication cost with negligible noise overhead