A Practical and Secure Byzantine Robust Aggregator

2025-06-29

AI Summary

In machine learning, data poisoning induces high-dimensional gradient anomalies—specifically, an ε-fraction of arbitrary, adaptive Byzantine-corrupted gradients. Method: We propose a quasi-linear-time robust mean estimation algorithm that requires no prior distributional assumptions or hand-tuned thresholds. It integrates adaptive outlier detection with Byzantine-resilient aggregation to achieve near-optimal bias bound O(ε√d) under ε-corruption. Contribution/Results: This is the first Byzantine-robust aggregator achieving both quasi-linear time complexity and statistically optimal accuracy. It is plug-and-play within standard distributed training pipelines. Experiments across ten canonical poisoning attacks demonstrate significant improvements in model robustness while maintaining high computational efficiency.

Technology Category

Machine Learning: Adversarial Learning & RobustnessComputer Vision: Adversarial Attacks & RobustnessNatural Language Processing: Safety and Robustness

Application Category

User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systemsSecurity and Privacy: Large-scale security measurementsResponsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

In machine learning security, one is often faced with the problem of removing outliers from a given set of high-dimensional vectors when computing their average. For example, many variants of data poisoning attacks produce gradient vectors during training that are outliers in the distribution of clean gradients, which bias the computed average used to derive the ML model. Filtering them out before averaging serves as a generic defense strategy. Byzantine robust aggregation is an algorithmic primitive which computes a robust average of vectors, in the presence of an $ε$ fraction of vectors which may have been arbitrarily and adaptively corrupted, such that the resulting bias in the final average is provably bounded. In this paper, we give the first robust aggregator that runs in quasi-linear time in the size of input vectors and provably has near-optimal bias bounds. Our algorithm also does not assume any knowledge of the distribution of clean vectors, nor does it require pre-computing any filtering thresholds from it. This makes it practical to use directly in standard neural network training procedures. We empirically confirm its expected runtime efficiency and its effectiveness in nullifying 10 different ML poisoning attacks.

Problem

• Computing robust average of high-dimensional vectors with outliers
• Provably bounding bias in final average despite corrupted vectors
• Enabling practical use in standard neural network training

Innovation

• Quasi-linear time Byzantine robust aggregator
• No prior knowledge of clean vector distribution
• Effective against diverse ML poisoning attacks