Medical Knowledge Is Not All You Need: When Medical Q&A Becomes Situated Patient Assistance

2026-09-27

AI Summary

This study addresses the reliability limitations of medical question-answering systems that rely solely on clinical knowledge while lacking situational context. Focusing on postoperative wound care, the authors conduct empirical user studies, LLM replay testing, and context injection experiments to reveal failure modes arising from the absence of visual and environmental awareness. The work proposes a design space for contextualized medical assistance, emphasizing dynamic information alignment between assistants and patients beyond traditional knowledge retrieval paradigms. Results demonstrate that 41.9% of user queries depend on non-procedural contextual information. Furthermore, even when augmented with clinical guidelines, most large language models still prematurely introduce subsequent steps or misinterpret unknown states, underscoring the critical role of situational awareness in enhancing the reliability of healthcare AI systems.

Abstract

Reliability in medical Q&A is often pursued by grounding responses in authoritative medical information. We show that when Q&A is embedded within ongoing care, reliability depends on more than what the system knows medically. In a study with 73 skin cancer patients practicing postoperative wound care, 41.9% of response-requiring questions depended on information beyond the procedure, including visual or physical state, environmental context, or prior actions. These demands varied across patients, consistent with patients recruiting the assistant into different informational roles. We then replayed the questions to seven LLMs while adding procedural and postoperative guidance. Errors remained substantial, including treating unknown states as known, even under explicit guardrails; with full procedural context, six of seven models more often introduced later steps prematurely. Based on these findings, we propose a design space for situated medical assistance that connects what the assistant and patient can each reliably establish to the form of assistance provided.

Problem

• Medical Q&A
• Situated assistance
• Large Language Models
• Reliability
• Postoperative care

Innovation

• Situated Medical Assistance
• Medical Q&A Reliability
• Large Language Models
• Context-aware Healthcare
• Design Space