Problem
Research questions and friction points this paper is trying to address.
Clinical Data Abstraction
Large Language Models
Information Extraction
AI Expertise
"I Know Where to Look," But Does the LLM? Charting the Gaps Between Clinical Expert Needs and Unstructured Data Abstraction Tools
🤖 AI Summary
研究通过与癌症研究团队合作设计并评估基于大语言模型的Libretto系统，解决临床数据提取中AI工具与专家需求之间的差距问题。
📝 Abstract
Clinical data abstraction, the process of distilling structured information from patient records, plays a key role in advancing knowledge about diseases such as cancer. Information extraction (IE) with large language models (LLMs) could accelerate this process, but it is unclear whether current frameworks effectively support clinical researchers without AI expertise. To address this, we co-designed an interactive LLM-based abstraction system called Libretto with seven cancer research teams, then evaluated the system's ability to help them answer real-world research questions. We found that while clinicians knew where and how to annotate complex concepts in patient notes, in twelve of fourteen tasks they faced barriers to replicating those intuitions with LLMs. Contextual note reliability judgments, difficulties in steering vibe-coded prompts, and inflexible evaluation strategies necessitated fundamental changes to the IE workflow. Our results highlight open problems for HCI research to bridge the gaps between AI data work tools and clinical users' needs.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Interactive LLM-based Abstraction System
Clinical Data Abstraction
Contextual Reliability Judgments
Steering Vibe-coded Prompts
🔎 Similar Papers
No similar papers found.
