Decentralized Quantile Regression for Feature-Distributed Massive Datasets with Privacy Guarantees

2025-04-23

AI Summary

This paper addresses privacy-preserving quantile regression in large-scale, feature-distributed settings over decentralized networks. Method: We propose DSG-cqr—the first decentralized surrogate gradient algorithm for conditional quantile regression that avoids conjugate gradient computation. It integrates convolution-type smoothing approximation with the Gaussian mechanism to achieve (ε,δ)-differential privacy (with ε ≤ 1) under feature partitioning, without requiring raw data sharing or global coordination. Auxiliary variables are introduced for residual estimation, and Wald statistics are employed to construct confidence intervals. Contribution/Results: We establish linear convergence of DSG-cqr to statistical accuracy. Empirical evaluations demonstrate its superior performance in estimation precision, differential privacy guarantees, and communication efficiency compared to existing approaches.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

In this paper, we introduce a novel decentralized surrogate gradient-based algorithm for quantile regression in a feature-distributed setting, where global features are dispersed across multiple machines within a decentralized network. The proposed algorithm, exttt{DSG-cqr}, utilizes a convolution-type smoothing approach to address the non-smooth nature of the quantile loss function. exttt{DSG-cqr} is fully decentralized, conjugate-free, easy to implement, and achieves linear convergence up to statistical precision. To ensure privacy, we adopt the Gaussian mechanism to provide $(epsilon,delta)$-differential privacy. To overcome the exact residual calculation problem, we estimate residuals using auxiliary variables and develop a confidence interval construction method based on Wald statistics. Theoretical properties are established, and the practical utility of the methods is also demonstrated through extensive simulations and a real-world data application.

Problem

• Decentralized quantile regression for feature-distributed massive datasets
• Privacy-preserving algorithm with Gaussian differential privacy guarantees
• Convolution smoothing for non-smooth quantile loss optimization

Innovation

• Decentralized surrogate gradient algorithm for quantile regression
• Convolution-type smoothing for non-smooth quantile loss
• Gaussian mechanism for differential privacy guarantees