Multimodal Forecasting of Sparse Intraoperative Hypotension Events Powered by Language Model

2025-05-28

AI Summary

Intraoperative hypotension (IOH) is highly prevalent and strongly associated with adverse outcomes, yet its prediction remains challenging due to event sparsity and difficulties in fusing heterogeneous multimodal data—particularly physiological time-series and structured clinical text. To address this, we propose IOHFuseLM, an end-to-end multimodal language model featuring a novel two-stage training paradigm: diffusion-augmented domain-adaptive pretraining followed by clinical-task fine-tuning. We further introduce a token-level alignment fusion mechanism that jointly encodes static clinical attributes as text and models dynamic physiological features synergistically. Evaluated on two real-world intraoperative datasets, IOHFuseLM achieves up to 8.2% higher AUC for IOH identification compared to state-of-the-art baselines, significantly improving early warning accuracy. The implementation is publicly available and designed for seamless integration into clinical decision-support systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Intraoperative hypotension (IOH) frequently occurs under general anesthesia and is strongly linked to adverse outcomes such as myocardial injury and increased mortality. Despite its significance, IOH prediction is hindered by event sparsity and the challenge of integrating static and dynamic data across diverse patients. In this paper, we propose extbf{IOHFuseLM}, a multimodal language model framework. To accurately identify and differentiate sparse hypotensive events, we leverage a two-stage training strategy. The first stage involves domain adaptive pretraining on IOH physiological time series augmented through diffusion methods, thereby enhancing the model sensitivity to patterns associated with hypotension. Subsequently, task fine-tuning is performed on the original clinical dataset to further enhance the ability to distinguish normotensive from hypotensive states. To enable multimodal fusion for each patient, we align structured clinical descriptions with the corresponding physiological time series at the token level. Such alignment enables the model to capture individualized temporal patterns alongside their corresponding clinical semantics. In addition, we convert static patient attributes into structured text to enrich personalized information. Experimental evaluations on two intraoperative datasets demonstrate that IOHFuseLM outperforms established baselines in accurately identifying IOH events, highlighting its applicability in clinical decision support scenarios. Our code is publicly available to promote reproducibility at https://github.com/zjt-gpu/IOHFuseLM.

Problem

• Predicting sparse intraoperative hypotension events accurately
• Integrating static and dynamic patient data effectively
• Enhancing clinical decision support for hypotension management

Innovation

• Two-stage training for sparse event prediction
• Token-level multimodal fusion of data
• Diffusion-augmented pretraining for sensitivity