MQFL-FHE: Multimodal Quantum Federated Learning Framework with Fully Homomorphic Encryption

2024-11-30

arXiv.org

AI Summary

In federated learning (FL), full homomorphic encryption (FHE) degrades model aggregation performance and impairs representation generalization. To address this, we propose the first privacy-preserving framework integrating multi-modal quantum Mixture of Experts (MQMoE) with FHE, enabling joint modeling of heterogeneous medical data—such as genomics and brain MRI. Innovatively, we incorporate quantum neural networks into FHE-enhanced FL and design a cross-modal alignment mechanism coupled with a sparse-gated MoE architecture, thereby improving feature disentanglement and long-tail class discrimination while preserving end-to-end privacy. Evaluated on multi-modal medical benchmarks, our method achieves an 8.3% improvement in classification accuracy, substantially mitigating FHE’s inherent precision loss. This work provides the first empirical validation that quantum enhancement can meaningfully improve the privacy–utility trade-off in encrypted FL.

Technology Category

• Machine Learning: Quantum Machine Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The integration of fully homomorphic encryption (FHE) in federated learning (FL) has led to significant advances in data privacy. However, during the aggregation phase, it often results in performance degradation of the aggregated model, hindering the development of robust representational generalization. In this work, we propose a novel multimodal quantum federated learning framework that utilizes quantum computing to counteract the performance drop resulting from FHE. For the first time in FL, our framework combines a multimodal quantum mixture of experts (MQMoE) model with FHE, incorporating multimodal datasets for enriched representation and task-specific learning. Our MQMoE framework enhances performance on multimodal datasets and combined genomics and brain MRI scans, especially for underrepresented categories. Our results also demonstrate that the quantum-enhanced approach mitigates the performance degradation associated with FHE and improves classification accuracy across diverse datasets, validating the potential of quantum interventions in enhancing privacy in FL.

Problem

• Mitigates performance drop from FHE in FL
• Enhances multimodal dataset representation
• Improves classification accuracy with quantum computing

Innovation

• Quantum-enhanced federated learning
• Multimodal quantum mixture experts
• Fully homomorphic encryption integration