Efficient Standardization of Clinical Notes using Large Language Models

2024-12-31

AI Summary

Clinical handwritten notes suffer from inconsistent handwriting, pervasive abbreviations, nonstandard terminology, grammatical and spelling errors, and disorganized formatting—severely hindering information extraction and interoperability in electronic health records (EHRs). To address this, we propose the first end-to-end LLM-based framework for clinical note standardization. It integrates terminology mapping, medical ontology alignment, and rule-augmented generative text reconstruction to jointly perform grammar/spelling correction, normalization of nonstandard clinical terms, abbreviation expansion, and structural reformatting—natively supporting interoperability standards such as FHIR. Evaluated on 1,618 real-world clinical notes, our method corrects on average 4.9 grammatical errors, 3.3 spelling errors, 3.1 nonstandard terms, and 15.8 abbreviations per note. Expert evaluation confirms high semantic fidelity and negligible information loss, significantly improving readability and downstream task performance.

Technology Category

• Natural Language Processing: Summarization
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Clinician notes are a rich source of patient information but often contain inconsistencies due to varied writing styles, colloquialisms, abbreviations, medical jargon, grammatical errors, and non-standard formatting. These inconsistencies hinder the extraction of meaningful data from electronic health records (EHRs), posing challenges for quality improvement, population health, precision medicine, decision support, and research. We present a large language model approach to standardizing a corpus of 1,618 clinical notes. Standardization corrected an average of $4.9 +/- 1.8$ grammatical errors, $3.3 +/- 5.2$ spelling errors, converted $3.1 +/- 3.0$ non-standard terms to standard terminology, and expanded $15.8 +/- 9.1$ abbreviations and acronyms per note. Additionally, notes were re-organized into canonical sections with standardized headings. This process prepared notes for key concept extraction, mapping to medical ontologies, and conversion to interoperable data formats such as FHIR. Expert review of randomly sampled notes found no significant data loss after standardization. This proof-of-concept study demonstrates that standardization of clinical notes can improve their readability, consistency, and usability, while also facilitating their conversion into interoperable data formats.

Problem

• Handwritten Notes Recognition
• Electronic Health Records Analysis
• Medical Information Extraction

Innovation

• Large-scale Language Models
• Medical Notes Standardization
• Health Information Quality Enhancement