Towards Adapting Open-Source Large Language Models for Expert-Level Clinical Note Generation

2024-04-25

AI Summary

To address the challenges of patient privacy preservation, on-premise deployment, and computational efficiency in clinical note generation, this paper proposes LLaMA-Clinic—a specialized system built upon LLaMA-2 13B. It introduces DistillDirect, a novel distillation framework integrating online policy-based reinforcement learning guided by Gemini 1.0 Pro as a teacher model, strictly enforcing predefined clinical formatting standards (i.e., format adherence is rule-governed, not model-autonomous). The system follows a three-stage adaptation pipeline: domain-adaptive pretraining, supervised fine-tuning, and reinforcement learning from AI/human feedback (RLED), augmented by clinical-domain corpus construction and structured format constraint modeling. Blind evaluation shows 90.4% of generated notes meet “acceptable” or higher quality thresholds; critically, the “Assessment and Plan” section achieves a real-world readiness score of 4.2/5—surpassing physician-written notes (4.1/5)—demonstrating the efficacy of a lightweight, regulatory-compliant, and high-fidelity specialty-level clinical generation system.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Proprietary Large Language Models (LLMs) such as GPT-4 and Gemini have demonstrated promising capabilities in clinical text summarization tasks. However, due to patient data privacy concerns and computational costs, many healthcare providers prefer using small, locally-hosted models over external generic LLMs. This study presents a comprehensive domain- and task-specific adaptation process for the open-source LLaMA-2 13 billion parameter model, enabling it to generate high-quality clinical notes from outpatient patient-doctor dialogues. Our process incorporates continued pre-training, supervised fine-tuning, and reinforcement learning from both AI and human feedback. We introduced a new approach, DistillDirect, for performing on-policy reinforcement learning with Gemini 1.0 Pro as the teacher model. Our resulting model, LLaMA-Clinic, can generate clinical notes comparable in quality to those authored by physicians. In a blinded physician reader study, the majority (90.4%) of individual evaluations rated the notes generated by LLaMA-Clinic as"acceptable"or higher across all three criteria: real-world readiness, completeness, and accuracy. In the more challenging"Assessment and Plan"section, LLaMA-Clinic scored higher (4.2/5) in real-world readiness than physician-authored notes (4.1/5). We highlight key considerations for future clinical note-generation tasks, emphasizing the importance of pre-defining a best-practice note format, rather than relying on LLMs to determine this for clinical practice.

Problem

• Adapting open-source LLMs for expert-level clinical note generation
• Overcoming privacy and cost barriers in healthcare with local models
• Ensuring clinical note quality comparable to physician-authored content

Innovation

• Domain-specific adaptation of LLaMA-2 model
• DistillDirect on-policy reinforcement learning
• Continued pre-training and human feedback