Is Clinical Text Enough? A Multimodal Study on Mortality Prediction in Heart Failure Patients

2026-04-02

AI Summary

This study addresses the challenge of accurately predicting short-term mortality risk in heart failure patients when relying solely on structured electronic health records. Leveraging a French heart failure cohort, the authors systematically evaluate various Transformer-based modeling strategies and propose a supervised multimodal fusion approach that effectively integrates entity-level representations from clinical text with structured variables. This method significantly outperforms conventional CLS embeddings and existing large language model (LLM) prompting techniques. Experimental results demonstrate that the proposed entity-aware multimodal Transformer achieves the best performance for short-term mortality prediction. In contrast, LLMs exhibit inconsistent performance across different input modalities and decoding strategies, with plain-text prompting yielding better results than structured or multimodal inputs.

Abstract

Accurate short-term mortality prediction in heart failure (HF) remains challenging, particularly when relying on structured electronic health record (EHR) data alone. We evaluate transformer-based models on a French HF cohort, comparing text-only, structured-only, multimodal, and LLM-based approaches. Our results show that enriching clinical text with entity-level representations improves prediction over CLS embeddings alone, and that supervised multimodal fusion of text and structured variables achieves the best overall performance. In contrast, large language models perform inconsistently across modalities and decoding strategies, with text-only prompts outperforming structured or multimodal inputs. These findings highlight that entity-aware multimodal transformers offer the most reliable solution for short-term HF outcome prediction, while current LLM prompting remains limited for clinical decision support.

Problem

• mortality prediction
• heart failure
• multimodal learning
• clinical text
• electronic health records

Innovation

• multimodal fusion
• entity-aware representation
• transformer-based models
• mortality prediction
• clinical text mining