MythTriage: Scalable Detection of Opioid Use Disorder Myths on a Video-Sharing Platform

2025-05-30

AI Summary

This study addresses the challenge of scalable detection of opioid use disorder (OUD)-related health misinformation on YouTube. We propose a three-tier collaborative triage framework integrating clinical experts, lightweight classification models, and large language models (LLMs). To support this, we construct the first clinically validated OUD misinformation video dataset, incorporating clinical knowledge-guided annotation, lightweight model inference, and LLM-driven active reasoning—augmented by an uncertainty-aware dynamic sample triage strategy. Evaluated on 2,900 search results and 343,000 recommended videos, our framework achieves a macro-F1 score of 0.86 while reducing annotation cost and time by over 76%. Our work quantifies OUD misinformation dissemination patterns and delivers a reproducible methodology and empirical evidence for public health interventions and platform-level content governance.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Misinformation & Fake News
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Responsible Web: Mitigating misinformation and disinformation
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Understanding the prevalence of misinformation in health topics online can inform public health policies and interventions. However, measuring such misinformation at scale remains a challenge, particularly for high-stakes but understudied topics like opioid-use disorder (OUD)--a leading cause of death in the U.S. We present the first large-scale study of OUD-related myths on YouTube, a widely-used platform for health information. With clinical experts, we validate 8 pervasive myths and release an expert-labeled video dataset. To scale labeling, we introduce MythTriage, an efficient triage pipeline that uses a lightweight model for routine cases and defers harder ones to a high-performing, but costlier, large language model (LLM). MythTriage achieves up to 0.86 macro F1-score while estimated to reduce annotation time and financial cost by over 76% compared to experts and full LLM labeling. We analyze 2.9K search results and 343K recommendations, uncovering how myths persist on YouTube and offering actionable insights for public health and platform moderation.

Problem

• Detecting opioid-use disorder myths on YouTube at scale
• Reducing annotation cost and time for misinformation labeling
• Analyzing myth prevalence and persistence in video-sharing platforms

Innovation

• Lightweight model for routine cases
• High-performing LLM for harder cases
• Reduces annotation time and cost