🤖 AI Summary
This study addresses the convergence challenges in federated zeroth-order optimization arising from client drift and sampling variance by proposing a direction-aggregation-based optimization framework. Methodologically, gradient-free updates are achieved through local trajectory averaging and server-side weighted aggregation, while two local sampling strategies are introduced to synergistically control estimation errors. Theoretically, rigorous convergence guarantees are established for the proposed method. Empirically, experiments on black-box adversarial attack tasks demonstrate that the framework effectively mitigates client drift, substantially reduces variance, and enhances overall optimization performance.
📝 Abstract
We study constrained nonsmooth nonconvex stochastic optimization in federated settings, where clients access only stochastic function evaluations. Existing federated zeroth-order methods primarily combine local zeroth-order updates with model averaging. However, they struggle with client drift induced by local projected updates and sampling variance in stochastic zeroth-order estimates. In this paper, we propose a federated zeroth-order framework based on direction aggregation, equipped with two local sampling schemes. Specifically, FedZOO constructs a local direction from a minibatch shared across multiple spherical queries, thereby controlling the stochastic errors arising from data sampling and gradient approximation. In contrast, FedVRZO forms its local estimator from independent sample--direction pairs, so that its sampling error is controlled directly by the number of sample--direction pairs. In both algorithms, clients compute the mean of the directions evaluated along their projected local trajectories, and the server performs a single projected update using the weighted aggregate of the client directions. Furthermore, we establish convergence guarantees for FedZOO and FedVRZO, respectively. Experiments on black-box adversarial attacks demonstrate the effectiveness of the proposed methods.