🤖 AI Summary
Clinical electronic health records (EHRs) contain critical anomalous events that evade detection by rule-based systems—particularly subtle, “apparently normal” events with substantial prognostic impact. Method: We propose an information-theoretic, foundation-model–driven anomaly detection framework that quantifies token- and event-level informativeness via context-aware entropy estimation and event saliency scoring. Contribution/Results: This work pioneers the integration of information theory into clinical event detection, overcoming reliance on handcrafted rules. By coupling informativeness scoring with interpretable attribution techniques, it provides event-level explanations for prognostic models. Experiments demonstrate that high-informativeness events identified by our method significantly improve downstream outcome prediction (average AUC gain of 0.08), while low-informativeness events can be safely pruned without performance degradation—validating the clinical utility and decision-support value of informativeness as a biomarker.
📝 Abstract
We present a foundation model-derived method to identify highly informative tokens and events in electronic health records. Our approach considers incoming data in the entire context of a patient's hospitalization and so can flag anomalous events that rule-based approaches would consider within a normal range. We demonstrate that the events our model flags are significant for predicting downstream patient outcomes and that a fraction of events identified as carrying little information can safely be dropped. Additionally, we show how informativeness can help interpret the predictions of prognostic models trained on foundation model-derived representations.