ReLMem: Learning Recurrent Memory for Longitudinal EHR Modeling

2026-09-29

AI Summary

This study addresses the escalating computational and memory overhead incurred when large language models (LLMs) process longitudinal electronic health records (EHRs) due to accumulating patient histories. To this end, we propose ReLMem, a framework built upon a frozen LLM that employs lightweight adapters to recursively update a fixed-capacity patient memory. Furthermore, it introduces a multi-granularity optimization strategy that integrates intermediate and predictive supervision signals, coupled with an attention alignment technique to effectively prevent the loss of critical historical information. Experimental results demonstrate that ReLMem reduces storage overhead by 97.1% while achieving performance comparable to full-history baselines. Notably, it improves the F1 score by approximately 4.7 percentage points over the strongest baseline, enabling efficient and accurate downstream clinical prediction.

Abstract

Longitudinal electronic health record (EHR) modeling requires integrating new visits with an expanding patient history. Yet the continual accumulation of clinical information imposes increasing computational and memory costs on large language models (LLMs) when they process and retain complete patient histories. A practical alternative is visit-wise recurrent compression, which incorporates each incoming visit into a compact, continually updated patient memory. However, under a fixed memory budget, successive updates must integrate new information without progressively losing critical historical evidence needed to subsequent tasks. To address this challenge, we introduce Recurrent Longitudinal Memory (ReLMem), a framework that learns to maintain fixed-capacity patient memory for efficient downstream prediction with a frozen LLM. ReLMem equips this LLM with lightweight compression adapters to recurrently update the memory from its previous state and each incoming visit, without rereading earlier records. Specifically, we develop a multi-granularity optimization strategy to preserve task-relevant information throughout recurrent updates and support downstream prediction from the final memory. The intermediate supervision aligns attention outputs from compressed memory and the full history under identical queries, while prediction supervision minimizes cross-entropy with ground truth answers conditioned on the final memory. On EHR-based medication prediction, ReLMem approaches the F1 scores of full-history baseline while reducing average retained historical storage by 97.1%. Under the same memory budget, it improves macro- and micro-F1 over the strongest compressed-memory baseline by 4.66 and 4.75 percentage points, respectively. These results highlight the value of learning recurrent patient memory for efficient longitudinal EHR modeling.

Problem

• Longitudinal EHR modeling
• Recurrent memory compression
• Fixed memory budget
• Information preservation
• Downstream prediction

Innovation

• Recurrent Memory
• Longitudinal EHR Modeling
• Compression Adapters
• Multi-granularity Optimization
• Large Language Models