Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity

2026-08-01

AI Summary

This work addresses the performance degradation in federated multimodal graph learning caused by the triple heterogeneity across tasks, modalities, and graph topologies. To tackle this challenge, the authors propose FedTCR, a novel framework that employs a two-stage paradigm to mitigate task heterogeneity and introduces a topology-aware cross-modal routing mechanism to jointly handle modality and topological heterogeneity. FedTCR further integrates structure-aware prototypes with a three-level contrastive learning strategy to achieve effective cross-client modality alignment while preserving discriminative representations. As the first systematic approach tailored for federated multimodal graph learning, FedTCR consistently outperforms existing methods across seven real-world datasets, achieving state-of-the-art performance on both graph-centric and modality-centric tasks.

Abstract

Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependencies, have been widely adopted across diverse domains. Federated multimodal graph learning (FMGL) extends federated graph learning (FGL) to MAGs, enabling collaborative optimization across decentralized MAGs without exposing raw data. However, naively applying existing FGL methods to FMGL is insufficient, as they fail to navigate the multifaceted heterogeneity inherent in decentralized MAGs, including task heterogeneity across diverse client objectives, modality heterogeneity from discrepant modality quality and semantic domains, and topology heterogeneity arising from divergent topological patterns with low cross-modality correlation. To address these challenges, we propose Federated multimodal graph learning with Topology-aware Cross-modal Routing (FedTCR), the first systematic algorithm designed for FMGL. To handle task heterogeneity, FedTCR employs a two-stage paradigm that comprises federated task-agnostic pre-training followed by isolated task-oriented fine-tuning. To jointly address modality and topology heterogeneity, FedTCR introduces a topology-aware cross-modal routing mechanism. Concretely, each client distills modality-specific knowledge into compact prototypes via topology-aware importance-weighted aggregation informed by graph structure; the server then evaluates cross-client cross-modal relationships among these structure-informed prototypes and routes informative ones as contrastive references, driving a tri-level cross-modal contrastive learning scheme that jointly aligns cross-client modalities while preserving discrimination. Experiments across 7 domains demonstrate that FedTCR outperforms state-of-the-art baselines on both graph-centric and modality-centric tasks.

Problem

• federated multimodal graph learning
• multifaceted heterogeneity
• task heterogeneity
• modality heterogeneity
• topology heterogeneity

Innovation

• Federated Multimodal Graph Learning
• Multifaceted Heterogeneity
• Topology-aware Cross-modal Routing
• Contrastive Learning
• Prototype Aggregation