First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection

2026-07-29

AI Summary

This study addresses the computational and storage bottlenecks induced by full-data training in distributed edge networks by investigating efficient and privacy-preserving methods for constructing robust core subsets. To this end, it introduces a novel tri-level optimization framework that formulates core subset selection as a tri-level problem with hierarchical constraints. The authors propose Federated First-order Constrained Tri-level Optimization (F²CTO), which integrates hierarchical composite value function reformulation with a distributed alternating projected gradient algorithm to solve the resulting problem. Theoretical analysis establishes that F²CTO achieves a non-asymptotic convergence rate of 𝒪(ε⁻³/²) to an ε-stationary point. Empirical evaluations demonstrate that the method is both efficient and effective in continual learning tasks requiring reliability and data efficiency.

Abstract

With the rapid advancement of the Internet of Things (IoT), massive amounts of data are generated across distributed edge networks. Training models on full data incurs significant computational overhead and storage bottlenecks, rendering coreset selection a critical paradigm. Furthermore, given the privacy-sensitive nature of local data and the escalating demand for model robustness in real-world deployments, developing an effective distributed optimization framework for robust coreset selection is vital, yet remains largely unexplored. To this end, this work first characterizes the hierarchical dependencies among coreset selection, robust optimization, and distributed learning, and formulates the distributed robust coreset selection as a trilevel optimization problem with level-wise constraints. Furthermore, to effectively solve the trilevel problem in a distributed manner, the \underline{F}ederated \underline{F}irst-order \underline{C}onstrained \underline{T}rilevel \underline{O}ptimization (F$^2$CTO) is proposed, which synergistically integrates a hierarchical composite value-function reformulation and a distributed alternating projected gradient algorithm. To the best of our knowledge, F$^2$CTO is the first method developed for distributed robust coreset selection, as well as the first distributed optimization approach for trilevel optimization problems with level-wise constraints. Additionally, we prove that the proposed method achieves a non-asymptotic convergence rate of $\mathcal{O}(ε^{-3/2})$ for finding an $ε$-stationary point. Extensive empirical evaluations on reliable continual learning demonstrate the effectiveness and efficiency of the proposed F$^2$CTO.

Problem

• distributed optimization
• robust coreset selection
• trilevel optimization
• edge networks
• privacy-sensitive data

Innovation

• Trilevel Optimization
• Distributed Learning
• Coreset Selection
• Robust Optimization
• First-order Method