Problem
Research questions and friction points this paper is trying to address.
Address asynchronous updates in distributed DRO environments
Effectively leverage prior distribution in optimization
Adjust robustness degree flexibly across scenarios
Federated Distributionally Robust Optimization with Non-Convex Objectives: Algorithm and Analysis
🤖 AI Summary
To address three key challenges in federated distributionally robust optimization (FDRO) for non-convex settings—difficulty in achieving convergence under asynchronous updates, insufficient exploitation of prior distributional knowledge, and lack of adaptive control over robustness levels—this paper proposes ASPIRE-EASE. The algorithm integrates asynchronous single-loop optimization, alternating gradient projection, and the iterative active-set method (EASE), coupled with a constraint-based D-norm uncertainty set. ASPIRE-EASE establishes the first theoretical convergence guarantee for non-convex FDRO and enables tunable trade-offs between robustness and model performance. Extensive experiments on real-world datasets demonstrate its rapid convergence, strong robustness against data heterogeneity and adversarial attacks, and superior generalization across diverse federated learning scenarios.
📝 Abstract
Distributionally Robust Optimization (DRO), which aims to find an optimal decision that minimizes the worst case cost over the ambiguity set of probability distribution, has been widely applied in diverse applications, e.g., network behavior analysis, risk management, etc. However, existing DRO techniques face three key challenges: 1) how to deal with the asynchronous updating in a distributed environment; 2) how to leverage the prior distribution effectively; 3) how to properly adjust the degree of robustness according to different scenarios. To this end, we propose an asynchronous distributed algorithm, named Asynchronous Single-looP alternatIve gRadient projEction (ASPIRE) algorithm with the itErative Active SEt method (EASE) to tackle the federated distributionally robust optimization (FDRO) problem. Furthermore, a new uncertainty set, i.e., constrained D-norm uncertainty set, is developed to effectively leverage the prior distribution and flexibly control the degree of robustness. Finally, our theoretical analysis elucidates that the proposed algorithm is guaranteed to converge and the iteration complexity is also analyzed. Extensive empirical studies on real-world datasets demonstrate that the proposed method can not only achieve fast convergence, and remain robust against data heterogeneity as well as malicious attacks, but also tradeoff robustness with performance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Asynchronous distributed ASPIRE algorithm
Constrained D-norm uncertainty set
Iterative Active Set method (EASE)
🔎 Similar Papers
No similar papers found.
