"What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets

2025-06-26

AI Summary

Understanding how users interact with LLM-based chatbots for health information retrieval—and identifying associated behavioral patterns and clinical risks—remains underexplored. Method: We curated HealthChat-11K, the first large-scale, multi-specialty (21 domains), clinically expert-annotated dataset of 11,000 real-world health consultation dialogues. We further proposed the first clinically grounded taxonomy for health consultation interactions, enabling multidimensional coding of contextual completeness, affective expression, and leading-question propensity. Contribution/Results: Analysis reveals that over 40% of dialogues lack critical clinical context; users frequently employ vague chief complaints, avoid self-diagnosis, and rely on guided questioning. We identified specific interaction patterns that trigger model sycophancy—a tendency to prioritize user agreement over clinical accuracy. All data and code are publicly released, establishing an empirical foundation and methodological framework for designing safe, trustworthy AI-driven healthcare interactions.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

People are increasingly seeking healthcare information from large language models (LLMs) via interactive chatbots, yet the nature and inherent risks of these conversations remain largely unexplored. In this paper, we filter large-scale conversational AI datasets to achieve HealthChat-11K, a curated dataset of 11K real-world conversations composed of 25K user messages. We use HealthChat-11K and a clinician-driven taxonomy for how users interact with LLMs when seeking healthcare information in order to systematically study user interactions across 21 distinct health specialties. Our analysis reveals insights into the nature of how and why users seek health information, such as common interactions, instances of incomplete context, affective behaviors, and interactions (e.g., leading questions) that can induce sycophancy, underscoring the need for improvements in the healthcare support capabilities of LLMs deployed as conversational AI. Code and artifacts to retrieve our analyses and combine them into a curated dataset can be found here: https://github.com/yahskapar/HealthChat

Problem

• Analyzing user behavior in seeking health information via LLMs
• Identifying risks in health-related chatbot conversations
• Improving healthcare support capabilities of conversational AI

Innovation

• Filtered large-scale datasets for HealthChat-11K
• Clinician-driven taxonomy for user interactions
• Analyzed 21 health specialties systematically