CoeF-SFL: Preserving Collaborative Server-Client Learning with Enhanced Communication Efficiency

2026-09-28

AI Summary

This study addresses the high communication overhead and optimization objective misalignment caused by auxiliary networks in split federated learning by proposing the CoeF-SFL framework. This method substantially reduces communication costs through single data exchange per round and gradient reuse, while eliminating auxiliary networks to preserve end-to-end optimization objectives. Furthermore, it introduces a curvature-based gradient compensation mechanism that leverages diagonal Hessian approximation and Jacobian-Hessian surrogate losses to correct stale gradients within the activation space. Experimental results demonstrate that the proposed framework significantly outperforms existing methods across both vision and language tasks. The source code has been made publicly available.

Abstract

Split Federated Learning (SFL) enables resource-constrained clients to participate in collaborative training, but vanilla SFL exchanges smashed data and gradients at every batch, which incurs significant communication overhead. Recent methods reduce this overhead with an auxiliary network at the client-side cut layer. However, we identify that this approach makes the client optimize a local objective that differs from the end-to-end objective, which fundamentally limits the collaborative training between the client and the server. We propose Compensated Feedback based SFL (CoeF-SFL), a communication-efficient framework that retains the end-to-end objective without any auxiliary network. In CoeF-SFL, the client and the server exchange the smashed data and the gradients once per round and reuse them during local training. Since this reuse makes the gradients stale on the client side, we compensate them with a curvature-based correction in the activation space and develop two variants. CoeF-D approximates the Hessian with a diagonal gradient outer product, while CoeF-J exploits the tractable Jacobian-based Hessian of a surrogate loss that upper-bounds the true loss. We provide the theoretical background of each method, characterizing its compensation. Across vision and language tasks, model capacities, cut layers, and data distributions, CoeF-SFL significantly outperforms auxiliary-network-based methods under the same communication frequency, and the improvement is most substantial on vision tasks. Code is available at https://anonymous.4open.science/r/CoeF-SFL-2686/README.md

Problem

• Split Federated Learning
• Communication Efficiency
• End-to-End Objective
• Collaborative Training

Innovation

• Split Federated Learning
• Communication Efficiency
• Curvature-based Correction
• Hessian Approximation
• End-to-End Objective