Refined Differentially Private Linear Regression via Extension of a Free Lunch Result

2026-04-10

AI Summary

This work proposes a novel method to enhance the accuracy and efficiency of linear regression under differential privacy constraints by leveraging a multidimensional simplex transformation. The approach maps bounded variables into a simplex space, enabling private summation to estimate the sufficient statistics required for regression without incurring additional privacy costs. This extends the “free lunch” principle—previously limited to univariate or unconstrained settings—to the multidimensional bounded case. Theoretical analysis and empirical evaluations demonstrate that, under the same privacy budget, the proposed method significantly outperforms existing differentially private linear regression techniques. Furthermore, the framework naturally generalizes to polynomial regression, offering a versatile and privacy-preserving solution for a broader class of regression problems.

Technology Category

Machine Learning: PrivacySearch and Optimization: Non-convex OptimizationReasoning under Uncertainty: Stochastic Optimization

Application Category

Security and Privacy: Data transparency and provenanceUser Modeling, Personalization and Recommendation: User privacy protection in personalized systemsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

As data-privacy regulations tighten and statistical models are increasingly deployed on sensitive human-sourced data, privacy-preserving linear regression has become a critical necessity. For the add-remove DP model, Kulesza et al. (2024) and Fitzsimons et al. (2024) have independently shown that the size of the dataset -- an important statistic for linear regression -- can be privately estimated for "free", via a simplex transformation of bounded variables and private sum queries on the transformed variables. In this work, we extend this free lunch result via carefully crafted multidimensional simplex transformations to variables and functions that are bounded in the interval [0,1]. We show that these transformations can be applied to refine the estimates of sufficient statistics needed for private simple linear regression based on ordinary least squares. We provide both analytical and numerical results to demonstrate the superiority of our approach. Our proposed transformations have general applicability and can be readily adapted for differentially private polynomial regression.

Problem

• differentially private linear regression
• add-remove DP
• sufficient statistics
• privacy-preserving estimation
• simplex transformation

Innovation

• differentially private linear regression
• simplex transformation
• free lunch result
• sufficient statistics
• polynomial regression