Verified Foundations for Differential Privacy

2024-12-02

IACR Cryptology ePrint Archive

AI Summary

Practical deployment of differential privacy (DP) suffers from security vulnerabilities arising from weak theoretical foundations, flawed random number generation, and floating-point arithmetic errors. To address these issues, we introduce SampCert—the first fully mechanized, scalable formal foundation for DP—implemented in the Lean theorem prover with over 12,000 lines of rigorously verified code. SampCert uniformly supports major DP variants, including pure DP, concentrated DP, and Rényi DP. We develop a general DP theory framework and present the first formal verification of discrete Laplace and Gaussian sampling algorithms, eliminating numerical inaccuracies. Furthermore, we design a lightweight probabilistic monad and novel compositional proof techniques that significantly improve formalization efficiency. Our verified primitives have been integrated into an AWS production-grade DP service, delivering high-assurance, deployable DP mechanisms. This work establishes a rigorous, trustworthy basis for privacy-preserving computation.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Probabilistic Programming
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Semantics and Knowledge: Provenance, trust, security and privacy, and ethical issues in managing semantic data

Abstract

Differential privacy (DP) has become the gold standard for privacy-preserving data analysis, but implementing it correctly has proven challenging. Prior work has focused on verifying DP at a high level, assuming the foundations are correct and a perfect source of randomness is available. However, the underlying theory of differential privacy can be very complex and subtle. Flaws in basic mechanisms and random number generation have been a critical source of vulnerabilities in real-world DP systems. In this paper, we present SampCert, the first comprehensive, mechanized foundation for differential privacy. SampCert is written in Lean with over 12,000 lines of proof. It offers a generic and extensible notion of DP, a framework for constructing and composing DP mechanisms, and formally verified implementations of Laplace and Gaussian sampling algorithms. SampCert provides (1) a mechanized foundation for developing the next generation of differentially private algorithms, and (2) mechanically verified primitives that can be deployed in production systems. Indeed, SampCert's verified algorithms power the DP offerings of Amazon Web Services (AWS), demonstrating its real-world impact. SampCert's key innovations include: (1) A generic DP foundation that can be instantiated for various DP definitions (e.g., pure, concentrated, R'enyi DP); (2) formally verified discrete Laplace and Gaussian sampling algorithms that avoid the pitfalls of floating-point implementations; and (3) a simple probability monad and novel proof techniques that streamline the formalization. To enable proving complex correctness properties of DP and random number generation, SampCert makes heavy use of Lean's extensive Mathlib library, leveraging theorems in Fourier analysis, measure and probability theory, number theory, and topology.

Problem

• Implementing differential privacy correctly is challenging due to complex underlying theory.
• Existing DP systems have vulnerabilities from flawed mechanisms and random number generation.
• SampCert provides a mechanized foundation for developing and verifying DP algorithms.

Innovation

• Comprehensive mechanized foundation for differential privacy
• Formally verified discrete Laplace and Gaussian sampling
• Simple probability monad with novel proof techniques