Quantifying Cross-Modal Interactions in Multimodal Glioma Survival Prediction via InterSHAP: Evidence for Additive Signal Integration

2026-03-31

AI Summary

This study investigates the authenticity and performance contribution of cross-modal synergy in multimodal glioma survival prediction. For the first time, the InterSHAP method is extended to Cox proportional hazards models and combined with variance decomposition and four fusion architectures to quantify the interaction strength between whole-slide images (WSI) and RNA-seq data on the TCGA-GBM/LGG datasets. Results reveal a negative correlation between predictive performance (C-index ranging from 0.64 to 0.82) and interaction strength (3.0%–4.8%). Modal contributions remain stable—approximately 40% from WSI, 55% from RNA-seq, and 4% from their interaction—indicating that model performance stems primarily from additive integration of complementary signals rather than synergistic learning. These findings offer a new interpretive paradigm for multimodal fusion in survival analysis.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Multimodal deep learning for cancer prognosis is commonly assumed to benefit from synergistic cross-modal interactions, yet this assumption has not been directly tested in survival prediction settings. This work adapts InterSHAP, a Shapley interaction index-based metric, from classification to Cox proportional hazards models and applies it to quantify cross-modal interactions in glioma survival prediction. Using TCGA-GBM and TCGA-LGG data (n=575), we evaluate four fusion architectures combining whole-slide image (WSI) and RNA-seq features. Our central finding is an inverse relationship between predictive performance and measured interaction: architectures achieving superior discrimination (C-index 0.64$\to$0.82) exhibit equivalent or lower cross-modal interaction (4.8\%$\to$3.0\%). Variance decomposition reveals stable additive contributions across all architectures (WSI${\approx}$40\%, RNA${\approx}$55\%, Interaction${\approx}$4\%), indicating that performance gains arise from complementary signal aggregation rather than learned synergy. These findings provide a practical model auditing tool for comparing fusion strategies, reframe the role of architectural complexity in multimodal fusion, and have implications for privacy-preserving federated deployment.

Problem

• multimodal fusion
• cross-modal interaction
• glioma survival prediction
• Cox proportional hazards
• Shapley interaction

Innovation

• InterSHAP
• multimodal fusion
• cross-modal interaction
• Cox model
• additive integration