TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification

2026-07-29

AI Summary

This work addresses the challenge in multimodal fusion where unknown modality reliability can lead to over-weighting of unreliable or redundant modalities, thereby degrading predictive performance. To mitigate this, the authors propose TIER-MoE, a novel mixture-of-experts architecture that, for the first time, jointly incorporates sample-level conditional modality risk—estimated via leave-one-out prediction loss—and expert subspace compatibility to enable risk-aware sparse modality-to-expert routing. The model retains a shared multimodal pathway to preserve complementary information across modalities. Evaluated on four biomedical multimodal datasets, TIER-MoE consistently outperforms state-of-the-art methods, achieving significant improvements in both Macro-F1 and Brier score, while also demonstrating strong zero-shot external generalization capabilities.

Abstract

The promise of multimodal fusion lies in combining complementary sources of evidence, yet more evidence does not always yield a better prediction. Recent multimodal models have advanced fusion through richer cross-modal interaction and sample-adaptive fusion. However, the influence assigned to a modality during fusion does not reveal whether that source is unreliable, redundant, or poorly matched to a specialized expert. To address this limitation, we introduce TIER-MoE, a risk-guided subspace mixture-of-experts model that defines sample-specific modality reliability as the prediction loss its unimodal predictor is expected to incur. This risk is learned from out-of-fold predictions generated by models that were not trained on the corresponding sample. TIER-MoE combines the estimated risk with expert-specific subspace compatibility for sparse modality-expert routing, while an always-active shared path preserves multimodal complementarity. We evaluate TIER-MoE on four public multimodal biomedical datasets spanning Alzheimer's disease status, skin-lesion malignancy, and retinal classification. Results demonstrate its superiority over state-of-the-art methods in predictive performance and probability calibration, with consistent improvements in Macro-F1 and Brier score and strong zero-shot generalization to an external cohort.

Problem

• multimodal fusion
• modality reliability
• biomedical classification
• expert routing
• risk estimation

Innovation

• conditional modality risk
• trust-informed routing
• mixture-of-experts
• multimodal fusion
• reliability estimation