PULSE-ICU: A Pretrained Unified Long-Sequence Encoder for Multi-task Prediction in Intensive Care Units

2025-11-27

AI Summary

ICU electronic health records (EHRs) exhibit high irregularity, heterogeneity, and temporal fragmentation, severely limiting the generalizability of clinical prediction models. To address this, we propose the first event-level self-supervised foundation model for ICU time series—requiring neither resampling nor handcrafted feature engineering. Our approach introduces an event-level unified embedding module that jointly encodes event type, numeric value, unit, and timestamp, coupled with a Longformer encoder to capture long-range temporal dependencies. The model enables low-parameter, cross-center and cross-task transfer learning. It achieves state-of-the-art performance across 18 diverse downstream tasks—including mortality prediction, intervention forecasting, and phenotyping—and demonstrates significant performance gains on three external datasets (eICU, HiRID, and P12), validating its strong generalizability and clinical adaptability.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Intensive care unit (ICU) data are highly irregular, heterogeneous, and temporally fragmented, posing challenges for generalizable clinical prediction. We present PULSE-ICU, a self-supervised foundation model that learns event-level ICU representations from large-scale EHR sequences without resampling or manual feature engineering. A unified embedding module encodes event identity, continuous values, units, and temporal attributes, while a Longformer-based encoder enables efficient modeling of long trajectories. PULSE-ICU was fine-tuned across 18 prediction tasks, including mortality, intervention forecasting, and phenotype identification, achieving strong performance across task types. External validation on eICU, HiRID, and P12 showed substantial improvements with minimal fine-tuning, demonstrating robustness to domain shift and variable constraints. These findings suggest that foundation-style modeling can improve data efficiency and adaptability, providing a scalable framework for ICU decision support across diverse clinical environments.

Problem

• Develops a self-supervised model for irregular ICU data
• Encodes diverse event attributes without manual feature engineering
• Enables multi-task prediction across various clinical outcomes

Innovation

• Self-supervised foundation model for ICU event representations
• Unified embedding module encoding event and temporal attributes
• Longformer-based encoder for efficient long-sequence modeling