Byzantine-Robust Distributed Sparse Learning Revisited

2026-05-13

AI Summary

This work addresses the vulnerability of high-dimensional sparse linear models to Byzantine attacks in distributed settings by proposing a unified robust learning framework. The framework integrates local ℓ₁-regularized robust estimation with a server-side robust aggregation mechanism, accommodating diverse tasks such as pseudo-Huber regression, quantile regression, and sparse support vector machines. Under mild conditions, the method achieves near-optimal non-asymptotic statistical convergence rates while maintaining communication efficiency. Notably, it provides the first unified Byzantine robustness guarantee for a broad class of sparse learning problems. Extensive simulations demonstrate that the proposed approach significantly outperforms existing methods in parameter estimation accuracy, support recovery, and classification performance.

Abstract

We revisit Byzantine robust distributed estimation for high-dimensional sparse linear models. By combining local $\ell_1$-regularized robust estimation with robust aggregation at the server, the framework applies to pseudo-Huber regression, quantile regression, and sparse SVM. We show that the resulting estimators yield non-asymptotic guarantees and attain near-optimal statistical rates under mild conditions, while remaining communication-efficient. Simulations confirm strong robustness in estimation, support recovery and classification accuracy under various Byzantine attacks.

Problem

• Byzantine robustness
• distributed learning
• sparse linear models
• high-dimensional estimation

Innovation

• Byzantine-robust
• distributed sparse learning
• robust aggregation
• high-dimensional statistics
• communication efficiency