CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment

2026-08-04

AI Summary

Clinical data are often overlooked in multimodal survival prediction due to their discrete, sparse, and low-dimensional nature, while inter-modal heterogeneity further hinders effective fusion. To address this, this work proposes a dual-level cross-modal interaction framework anchored on clinical information: first, clinical tabular data are embedded into a high-dimensional space using a pretrained model; then, local prototype associations—mediated by cross-attention—and global feature alignment—enforced via maximum mean discrepancy (MMD)—are jointly leveraged to integrate histopathology, genomic, and clinical modalities. Evaluated across five TCGA cancer cohorts, the proposed method significantly improves survival prediction performance, achieving state-of-the-art results.

Abstract

Multimodal learning has significantly advanced survival prediction by integrating pathology images with genomic data. However, clinical information, despite its critical role in reflecting a patient' s overall health, remains underutilized due to its discrete, sparse, and low-dimensional nature. Furthermore, the inherent heterogeneity across these modalities pose significant challenges in modeling cross-modal interactions. In this paper, we propose CIGTSurv, a Clinical Information Guided Tri-modal framework for Survival prediction. Specifically, we first design a holistic text template and use pretrained foundation models to transform clinical tabular data into high-dimensional tokenized embeddings. Using clinical information as an anchor, we then introduce a dual-level interaction mechanism: 1) a local prototype association (LPA) module based on cross-attention to explicitly learn token-level correspondences between different modalities, and 2) a global feature alignment (GFA) loss based on Maximum Mean Discrepancy (MMD) to implicitly enhance cross-modal distribution consistency. Extensive experiments on five TCGA cancer cohorts demonstrate that CIGTSurv achieves state-of-the-art (SOTA) survival prediction performance. Our source code is publicly available at https://github.com/Daijing-ai/CIGT-Surv.git.

Problem

• survival prediction
• multimodal learning
• clinical information
• cross-modal interaction
• heterogeneity

Innovation

• Tri-modal learning
• Clinical information integration
• Local Prototype Association
• Global Feature Alignment
• Survival prediction