HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction

2026-03-17

AI Summary

The high cost and scarcity of protein expression data have hindered the joint utilization of histopathological images and protein biomarkers for cancer prognosis. To address this challenge, this work proposes the HGP-Mamba framework, which first leverages a pretrained foundation model to generate protein embeddings from whole-slide images (WSIs), enabling data-efficient extraction of protein-level features. It then introduces two novel modules—Local Interaction-aware Mamba (LiAM) and Global Interaction-enhanced Mamba (GiEM)—to facilitate fine-grained and holistic multimodal fusion. Evaluated on four public cancer datasets, the proposed method significantly outperforms existing approaches, achieving state-of-the-art performance in both predictive accuracy and computational efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Recent advances in multimodal learning have significantly improved cancer survival risk prediction. However, the joint prognostic potential of protein markers and histopathology images remains underexplored, largely due to the high cost and limited availability of protein expression profiling. To address this challenge, we propose HGP-Mamba, a Mamba-based multimodal framework that efficiently integrates histological with generated protein features for survival risk prediction. Specifically, we introduce a protein feature extractor (PFE) that leverages pretrained foundation models to derive high-throughput protein embeddings directly from Whole Slide Images (WSIs), enabling data-efficient incorporation of molecular information. Together with histology embeddings that capture morphological patterns, we further introduce the Local Interaction-aware Mamba (LiAM) for fine-grained feature interaction and the Global Interaction-enhanced Mamba (GiEM) to promote holistic modality fusion at the slide level, thus capture complex cross-modal dependencies. Experiments on four public cancer datasets demonstrate that HGP-Mamba achieves state-of-the-art performance while maintaining superior computational efficiency compared with existing methods. Our source code is publicly available at <a href="https://github.com/Daijing-ai/HGP-Mamba.git">this https URL</a>.

Problem

• survival risk prediction
• histopathology
• protein markers
• multimodal learning
• cancer prognosis

Innovation

• Mamba
• multimodal fusion
• protein feature generation
• histopathology
• survival prediction