Anchor-driven Multi-modal Multi-scale Expert Selection for Survival Prediction

2026-10-05

AI Summary

This study addresses the challenges of missing multimodal feature alignment and spatial heterogeneity in pathological images that hinder cancer survival prediction by proposing the AM²ES framework. Methodologically, it introduces learnable semantic anchors to drive structured semantic alignment between transcriptomic data and pathological images. Additionally, a Hierarchical Mixture-of-Experts (H-MoE) module is designed to simulate the clinical pathological diagnostic workflow, decoupling intra-scale region filtering from inter-scale hierarchical routing to adaptively identify critical tissue scales. The proposed framework achieves state-of-the-art performance across multiple TCGA cancer cohorts. Furthermore, it provides fine-grained interpretability by visualizing how molecular pathways influence expert routing decisions.

Abstract

The integrative analysis of histopathological Whole-Slide Images (WSIs) and transcriptomic profiles holds significant promise for cancer survival prediction. However, existing methods typically project multi-modal features directly into a shared latent space without explicit alignment, leading to the entanglement of mismatched morphological cues and molecular signals. Furthermore, current fusion strategies often treat the extreme spatial heterogeneity of WSIs uniformly, lacking mechanisms to adaptively prioritize clinically relevant tissue scales for individual patients. To address these limitations, we propose an Anchor-driven Multi-modal Multi-scale Expert Selection (AM$^2$ES) framework for survival prediction. Specifically, we present an Anchor-driven Multi-modal Fusion (AMF) module, which introduces learnable semantic anchors as cross-modal mediators to bridge the semantic gap by enforcing a structurally regularized alignment between transcriptomic features and multi-scale pathology representations. Built upon this aligned semantic space, we further design a Hierarchical Mixture-of-Experts (H-MoE) selection module to decouple the hierarchical prognostic selection process. Mimicking the pathologist's diagnostic workflow, H-MoE performs (i) Intra-scale Expert Filtering to discriminatively identify salient tumor regions within each magnification, and (ii) Inter-scale Hierarchy Routing to dynamically weight and select the most informative resolution levels. Extensive experiments on multiple TCGA cancer cohorts demonstrate that our AM$^2$ES achieves state-of-the-art performance while offering fine-grained interpretability by visualizing how specific molecular pathways drive the expert routing decisions across tissue scales. The code will be released at https://github.com/taozh2017/AM2ES.

Problem

• Survival Prediction
• Multi-modal Fusion
• Whole-Slide Images
• Spatial Heterogeneity
• Cross-modal Alignment

Innovation

• Multi-modal Fusion
• Mixture-of-Experts
• Survival Prediction
• Whole-Slide Images
• Semantic Anchors