FL-APU: A Software Architecture to Ease Practical Implementation of Cross-Silo Federated Learning

2024-09-17

2024 2nd International Conference on Federated Learning Technologies and Applications (FLTA)

AI Summary

Addressing the practical deployment challenges of cross-institutional federated learning (FL)—particularly concerning resource coordination, lack of mutual trust, process opacity, and insufficient production-readiness—this paper proposes a scenario-driven software architecture enabling trustworthy multi-enterprise collaborative modeling. Methodologically, it introduces: (i) a novel dynamic admission mechanism integrating governance policies with identity authentication; (ii) role-based access control coupled with blockchain-anchored immutable logging to ensure full traceability and auditability of the entire training lifecycle; and (iii) a scenario-aware API orchestration framework and a formalized federated governance protocol model. Empirically validated in cross-organizational healthcare settings, the architecture demonstrates significant improvements in trustworthiness, regulatory compliance adaptability, and operational maintainability. It constitutes the first production-grade architectural solution bridging institutional data silos in FL.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Adversarial Agents

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Data transparency and provenance

Abstract

Federated Learning (FL) is an upcoming technology that is increasingly applied in real-world applications. Early applications focused on cross-device scenarios, where many participants with limited resources train machine learning (ML) models together, e.g., in the case of Google's GBoard. Contrarily, cross-silo scenarios have only few participants but with many resources, e.g., in the healthcare domain. Despite such early efforts, FL is still rarely used in practice and best practices are, hence, missing. For new applications, in our case inter-organizational cross-silo applications, overcoming this lack of role models is a significant challenge. In order to ease the use of FL in real-world cross-silo applications, we here propose a scenario-based architecture for the practical use of FL in the context of multiple companies collaborating to improve the quality of their ML models. The architecture emphasizes the collaboration between the participants and the FL server and extends basic interactions with domain-specific features. First, it combines governance with authentication, creating an environment where only trusted participants can join. Second, it offers traceability of governance decisions and tracking of training processes, which are also crucial in a production environment. Beyond presenting the architectural design, we analyze requirements for the real-world use of FL and evaluate the architecture with a scenario-based analysis method.

Problem

• Federated Learning
• Cross-Company Collaboration
• Resource Allocation

Innovation

• Federated Learning
• Trust Mechanism
• Resource-limited Entities