Can Textual Gradient Work in Federated Learning?

2025-02-27

AI Summary

This work addresses the challenge that conventional federated learning (FL) frameworks face in handling textual feedback—where no explicit numerical loss function is available. To this end, we propose FedTextGrad, the first FL paradigm grounded in *textual gradients*. In this framework, clients optimize prompts using only textual feedback, while the server performs text-level aggregation to coordinate model updates—thereby eliminating reliance on numerical gradients or differentiable loss functions. A key innovation is the incorporation of the Uniform Information Density principle to guide large language model (LLM)-based prompt aggregation, substantially enhancing information fidelity across distributed textual updates. Extensive experiments validate the feasibility of textual gradients in FL, characterize the impact of critical hyperparameters (e.g., local update steps), and demonstrate significant performance gains on downstream tasks.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent studies highlight the promise of LLM-based prompt optimization, especially with TextGrad, which automates differentiation'' via texts and backpropagates textual feedback. This approach facilitates training in various real-world applications that do not support numerical gradient propagation or loss calculation. In this paper, we systematically explore the potential and challenges of incorporating textual gradient into Federated Learning (FL). Our contributions are fourfold. Firstly, we introduce a novel FL paradigm, Federated Textual Gradient (FedTextGrad), that allows clients to upload locally optimized prompts derived from textual gradients, while the server aggregates the received prompts. Unlike traditional FL frameworks, which are designed for numerical aggregation, FedTextGrad is specifically tailored for handling textual data, expanding the applicability of FL to a broader range of problems that lack well-defined numerical loss functions. Secondly, building on this design, we conduct extensive experiments to explore the feasibility of FedTextGrad. Our findings highlight the importance of properly tuning key factors (e.g., local steps) in FL training. Thirdly, we highlight a major challenge in FedTextGrad aggregation: retaining essential information from distributed prompt updates. Last but not least, in response to this issue, we improve the vanilla variant of FedTextGrad by providing actionable guidance to the LLM when summarizing client prompts by leveraging the Uniform Information Density principle. Through this principled study, we enable the adoption of textual gradients in FL for optimizing LLMs, identify important issues, and pinpoint future directions, thereby opening up a new research area that warrants further investigation.

Problem

• Textual gradient integration in Federated Learning.
• FedTextGrad handles textual data in FL.
• Retaining essential information in prompt aggregation.

Innovation

• Textual Gradient in FL
• FedTextGrad for textual data
• Uniform Information Density principle