Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights

2025-09-05

AI Summary

The integration of federated learning (FL) with foundation models (FMs) lacks a systematic survey and unified taxonomic framework, hindering collaborative modeling over distributed private data. Method: We propose the first comprehensive, lifecycle-oriented taxonomy for FL–FM integration, unifying technical paradigms—including self-supervised learning, fine-tuning, knowledge distillation, and transfer learning—and systematically categorizing 42 representative methods. Our analysis draws on bibliometric and technical reviews of 250 core papers selected from an initial pool of 4,200+. Contribution/Results: We introduce a three-dimensional evaluation framework assessing scalability, efficiency, and complexity. The resulting self-consistent knowledge system provides reproducible technical roadmaps and practical guidelines—particularly for high-privacy domains such as healthcare—thereby advancing both theoretical understanding and real-world deployment of privacy-preserving foundation model learning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated learning has the potential to unlock siloed data and distributed resources by enabling collaborative model training without sharing private data. As more complex foundational models gain widespread use, the need to expand training resources and integrate privately owned data grows as well. In this article, we explore the intersection of federated learning and foundational models, aiming to identify, categorize, and characterize technical methods that integrate the two paradigms. As a unified survey is currently unavailable, we present a literature survey structured around a novel taxonomy that follows the development life-cycle stages, along with a technical comparison of available methods. Additionally, we provide practical insights and guidelines for implementing and evolving these methods, with a specific focus on the healthcare domain as a case study, where the potential impact of federated learning and foundational models is considered significant. Our survey covers multiple intersecting topics, including but not limited to federated learning, self-supervised learning, fine-tuning, distillation, and transfer learning. Initially, we retrieved and reviewed a set of over 4,200 articles. This collection was narrowed to more than 250 thoroughly reviewed articles through inclusion criteria, featuring 42 unique methods. The methods were used to construct the taxonomy and enabled their comparison based on complexity, efficiency, and scalability. We present these results as a self-contained overview that not only summarizes the state of the field but also provides insights into the practical aspects of adopting, evolving, and integrating foundational models with federated learning.

Problem

• Integrating foundational models with federated learning paradigms
• Surveying technical methods for collaborative private data training
• Providing practical guidelines for healthcare domain implementation

Innovation

• Combining federated learning with foundational models
• Surveying methods via development lifecycle taxonomy
• Providing practical implementation guidelines for healthcare