An Interactive Framework for Finding the Optimal Trade-off in Differential Privacy

2025-09-04

AI Summary

This work addresses the optimal trade-off between privacy budget and model accuracy in differential privacy. Methodologically, it introduces an interactive multi-objective optimization framework: (i) leveraging the structural property that accuracy is directly optimizable under a fixed privacy budget, the Pareto frontier is theoretically derived and explicitly modeled; (ii) replacing conventional pairwise comparisons with a user interaction mechanism based on hypothesized trade-off curves to improve preference learning efficiency; and (iii) integrating theoretical modeling with Bayesian optimization to jointly learn the multi-objective Pareto front and user preferences. Experiments across logistic regression and deep transfer learning tasks on six real-world datasets demonstrate that the method achieves significantly faster convergence to the optimal privacy–accuracy balance—requiring fewer user interactions and lower computational overhead—compared to existing approaches.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Differential privacy (DP) is the standard for privacy-preserving analysis, and introduces a fundamental trade-off between privacy guarantees and model performance. Selecting the optimal balance is a critical challenge that can be framed as a multi-objective optimization (MOO) problem where one first discovers the set of optimal trade-offs (the Pareto front) and then learns a decision-maker's preference over them. While a rich body of work on interactive MOO exists, the standard approach -- modeling the objective functions with generic surrogates and learning preferences from simple pairwise feedback -- is inefficient for DP because it fails to leverage the problem's unique structure: a point on the Pareto front can be generated directly by maximizing accuracy for a fixed privacy level. Motivated by this property, we first derive the shape of the trade-off theoretically, which allows us to model the Pareto front directly and efficiently. To address inefficiency in preference learning, we replace pairwise comparisons with a more informative interaction. In particular, we present the user with hypothetical trade-off curves and ask them to pick their preferred trade-off. Our experiments on differentially private logistic regression and deep transfer learning across six real-world datasets show that our method converges to the optimal privacy-accuracy trade-off with significantly less computational cost and user interaction than baselines.

Problem

• Optimizing privacy-accuracy trade-off in differential privacy
• Modeling Pareto front for multi-objective optimization
• Reducing computational cost and user interaction

Innovation

• Directly models Pareto front for efficiency
• Replaces pairwise comparisons with curve selection
• Reduces computational cost and user interaction