Resilience of Rademacher chaos of low degree

2024-02-16

AI Summary

This work investigates the robustness of low-order (≥2) Rademacher chaos under adversarial sign flips—specifically, the maximum number of flips tolerable while preserving the largest atomic probability. We propose the first instance-dependent, decoupled lower bound on Rademacher chaos resilience, and for the first time systematically distinguish the asymptotic robustness behaviors of second-order versus higher-order chaos. To achieve this, we develop a novel analytical framework tailored to tensor structure, integrating Dudley’s maximal inequality, the Hanson–Wright inequality (for second-order chaos), the Adamczak–Wolff tensor concentration inequality (for higher-order chaos), and the Kolmogorov–Rogozin anti-concentration inequality. Our analysis yields tight resilience bounds for second-order chaos under both sparse and dense matrix settings, and extends these results to arbitrary constant-order higher-order chaos. Crucially, all bounds are efficiently computable, yielding practical, instance-specific robustness guarantees.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Measurement, analysis, and circumvention of Web censorship
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The resilience of a Rademacher chaos is the maximum number of adversarial sign-flips that the chaos can sustain without having its largest atom probability significantly altered. Inspired by probabilistic lower-bound guarantees for the resilience of linear Rademacher chaos, obtained by Bandeira, Ferber, and Kwan (Advances in Mathematics, Vol. $319$, $2017$), we provide probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary yet sufficiently low degree. Our main results distinguish between Rademacher chaos of order two and those of higher order. In that, our first main result pertains to the resilience of decoupled bilinear Rademacher forms where different asymptotic behaviour is observed for sparse and dense matrices. For our second main result, we bootstrap our first result in order to provide resilience guarantees for quadratic Rademacher chaos. Our third main result, generalises the first and handles the resilience of decoupled Rademacher chaos of arbitrary yet sufficiently low order. Our results for decoupled Rademacher chaos of order two and that of higher order whilst are established through the same conceptual framework, differ substantially. A difference incurred due to the implementation of the same conceptual argument. The order two result is established using Dudley's maximal inequality for sub-Gaussian processes, the Hanson-Wright inequality, as well as the Kolmogorov-Rogozin inequality. To handle higher order chaos, appeals to Dudley's inequality as well as the Hanson-Wright inequality are replaced with tools suited for random tensors. Appeals to the Hanson-Wright inequality are replaced with appeals to a concentration result for random tensors put forth by Adamczak and Wolff. Our results are instance-dependent and thus allow for the efficient computation of resilience guarantees provided the order of the chaos is constant.

Problem

• Study resilience of Rademacher chaos under adversarial sign-flips.
• Provide probabilistic lower-bound guarantees for low-degree chaos.
• Distinguish resilience behavior between second-order and higher-order chaos.

Innovation

• Probabilistic lower-bound guarantees for Rademacher chaos resilience.
• Different asymptotic behavior for sparse and dense matrices.
• Tools for random tensors replace traditional inequalities.