MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

2026-07-16

AI Summary

Existing diffusion-based approaches for multimodal knowledge graph completion directly denoise raw features, struggling to simultaneously address relation-dependent clue selection, cross-modal semantic alignment, and structure-aware generation, which often leads to semantic inconsistency and suboptimal performance. To overcome these limitations, this work proposes a novel “align-then-diffuse” paradigm. It first employs a Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE) to select relevant modal pathways and leverages a frozen multimodal large language model (MLLM) to achieve cross-modal semantic alignment. Subsequently, diffusion-based generation conditioned on the knowledge graph structure (KGDT) is performed in a unified latent space. By decoupling semantic integration from generation, the proposed method significantly outperforms strong baselines across three benchmark datasets, effectively enhancing both generative consistency and completion accuracy.

Abstract

Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such a design forces the denoiser to simultaneously perform relation-dependent cue selection, cross-modal semantic alignment, and structure-aware entity generation, which introduces noisy and semantically inconsistent conditions for diffusion and consequently leads to suboptimal completion performance. To address this limitation, we propose MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts (MGDT), a novel MKGC framework built on an align-then-diffuse paradigm. MGDT first employs a Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE) module to select relation-relevant multimodal semantic transformation paths and suppress irrelevant modality interference. MGDT then uses a frozen Multimodal Large Language Model (MLLM) as a semantic anchor to align the routed multimodal representations into a unified latent space and reduce cross-modal semantic heterogeneity. Finally, a Knowledge Graph Diffusion Transformer (KGDT) performs graph-conditioned denoising generation in the aligned space to produce the missing entity representation. Experiments on three benchmark datasets show that MGDT consistently outperforms strong baselines.

Problem

• Multimodal Knowledge Graph Completion
• Diffusion Model
• Cross-modal Semantic Alignment
• Relation-dependent Cue Selection
• Semantic Heterogeneity

Innovation

• Relation-Adaptive Mixture-of-Experts
• MLLM-Guided Alignment
• Align-then-Diffuse Paradigm
• Multimodal Knowledge Graph Completion
• Diffusion Transformer