Separating Clicks from Baits: Using Large Language Models to Detect Misleading YouTube Thumbnails

2026-07-26

AI Summary

This study addresses the growing concern of misleading thumbnails on YouTube, which significantly undermine user trust and content credibility. To tackle this issue, the authors propose a multimodal detection approach that integrates video descriptions, thumbnail images, and subtitle text. They construct the first large-scale, cross-cultural dataset for this task and conduct a systematic evaluation of both proprietary (e.g., GPT-4o, Claude 3.5 Sonnet) and open-source (e.g., LLaVA-v1.5, Qwen2.5-VL) multimodal large language models. Experimental results demonstrate that Claude 3.5 Sonnet achieves a scene-specific accuracy of 93.8%, with both precision and recall exceeding 92%, highlighting the efficacy and feasibility of leveraging large multimodal models for detecting misleading content. This work establishes a new paradigm for multimodal misinformation detection in real-world video platforms.

Abstract

Misleading video thumbnails on platforms like YouTube are a pervasive problem, undermining user trust and platform integrity. This paper proposes a novel multi-modal detection pipeline that uses Large Language Models (LLMs) to flag misleading thumbnails. We first construct a comprehensive dataset of 2,843 videos from eight countries, including 1,359 misleading thumbnail videos that collectively amassed over 7.6 billion views, providing a unique cross-cultural perspective on this global issue. Our detection pipeline integrates video-to-text descriptions, thumbnail images, and subtitle transcripts to holistically analyze content and flag misleading thumbnails. Through extensive experimentation and prompt engineering, we evaluate the performance of four frontier-level LLMs, including GPT-4o, GPT-4o Mini, Claude 3.5 Sonnet, and Gemini-1.5 Flash. We further evaluate open-weight vision-language models, LLaVA-v1.5 and Qwen2.5-VL-7B-Instruct, to assess the generalizability of our approach beyond proprietary systems. Our findings show the effectiveness of LLMs in identifying misleading thumbnails, with Claude 3.5 Sonnet consistently showing strong performance, achieving an accuracy of 93.8%, precision over 92%, and recall exceeding 94% in certain scenarios. Beyond evaluating detection performance, we conducted a careful failure analysis to understand when LLMs fail in identifying misleading thumbnails. We discuss the implications of our findings for content moderation, user experience, and the ethical considerations of deploying such systems at scale. Our findings pave the way for more transparent, trustworthy video platforms and stronger content integrity for audiences worldwide.

Problem

• misleading thumbnails
• YouTube
• content integrity
• user trust
• clickbait

Innovation

• misleading thumbnails
• large language models
• multimodal detection
• content moderation
• cross-cultural dataset