🤖 AI Summary
The scarcity of Italian clinical text severely hinders the application of large language models in emergency medicine. To address this gap, this work introduces the EDEN corpus, comprising approximately 4 million fully anonymized Italian emergency department clinical notes. A subset of 6,000 notes was annotated by clinical experts using a structured case report form encompassing 132 criteria, with a focus on two acute conditions: dyspnea and loss of consciousness. This study presents the largest publicly available Italian clinical note dataset to date and proposes a novel benchmark task for structured information extraction framed as case report form completion. Leveraging an on-site anonymization pipeline and a multi-expert iterative annotation protocol, the authors conduct zero-shot evaluations using Gemma-27B and MedGemma-27B, substantially mitigating the data scarcity that has impeded the development and deployment of medical large language models for Italian.
📝 Abstract
We present EDEN (Emergency Department Electronic Notes), a new and unique large-scale corpus of clinical notes produced in Emergency Departments of Italian hospitals. The corpus, in its current version, is composed of approximately 4 million clinical notes fully anonymized, covering diverse phases of patient care during the stay in the emergency department. In addition, a subset of about six thousand notes has been manually annotated by clinical experts through a structured Case Report Form (CRF) containing 132 items relevant for two patient situations in emergency departments, dyspnea and loss of consciousness. Items may assume numerical values (e.g., for blood saturation), categorical (e.g., for level of consciousness ), binary (e.g., for presence of traumas), and mixed value types. The annotation process involved multiple clinicians and underwent iterative revision to resolve ambiguities in item formulation, resulting in a richly structured (although high imbalanced) resource. The dataset aims to fill a relevant gap of data able to support both the development and the use of Large Language Models in concrete medical applications. We describe the data collection protocol, the on-site anonymisation pipeline, corpus statistics, and the annotation scheme. Finally, we propose CRF-filling as a novel structured information extraction benchmark, and provide zero-shot baseline resulting from Gemma-27B and MedGemma-27B. To the best of our knowledge, the EDEN dataset is the largest freely available corpus of clinical notes existing for the Italian language.