Overview of the MEDIQA-OE 2025 Shared Task on Medical Order Extraction from Doctor-Patient Consultations

2025-10-30

AI Summary

This study addresses the novel task of automatic executable medical instruction extraction from physician–patient dialogues, aiming to alleviate clinical documentation burden and enhance electronic health record (EHR) automation. We formally define the task for the first time, construct the inaugural annotated dataset specifically for medical instruction extraction, and organize the international MedIE 2024 shared task—thereby bridging a critical gap in converting spoken or textual clinical interactions into structured, actionable medical directives. Methodologically, we integrate closed- and open-weight large language models, synergizing natural language understanding, fine-grained information extraction, and dialogue state modeling to support multi-stage instruction identification and standardized output generation. Six international teams participated in MedIE 2024, submitting diverse solutions and establishing a public benchmark leaderboard. Empirical evaluation on real-world clinical data validates the effectiveness and clinical applicability of the proposed approaches, advancing medical NLP toward operationally actionable outcomes.

Technology Category

• Natural Language Processing: Information Extraction
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Clinical documentation increasingly uses automatic speech recognition and summarization, yet converting conversations into actionable medical orders for Electronic Health Records remains unexplored. A solution to this problem can significantly reduce the documentation burden of clinicians and directly impact downstream patient care. We introduce the MEDIQA-OE 2025 shared task, the first challenge on extracting medical orders from doctor-patient conversations. Six teams participated in the shared task and experimented with a broad range of approaches, and both closed- and open-weight large language models (LLMs). In this paper, we describe the MEDIQA-OE task, dataset, final leaderboard ranking, and participants' solutions.

Problem

• Extracting medical orders from doctor-patient conversations
• Converting conversations into actionable EHR orders
• Reducing clinical documentation burden through automation

Innovation

• Extracting medical orders from doctor-patient conversations
• Using large language models for clinical documentation
• Reducing documentation burden through automated order extraction