MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes

2026-07-30

AI Summary

This study addresses the limitations of current large vision-language models (LVLMs) in interpreting internet memes that rely on cultural context, where inadequate integration of visual content and external knowledge often leads to incomplete semantic understanding. To this end, the authors introduce MemeBench—the first fine-grained diagnostic benchmark specifically designed for Chinese and English subcultural memes—and propose the VIKR framework (Visual cues, Identity linkage, Knowledge units, Reasoning mechanisms) to systematically evaluate model capabilities. Leveraging a newly constructed CultureBase knowledge repository, they develop an entity-guided Knowledge-Augmented Retrieval (KAR) method. Experiments reveal a 22.6% average gap in visual-knowledge comprehension across 26 mainstream LVLMs; KAR improves VIKR success rates by 3.6–7.4% on four representative models, effectively mitigating knowledge deficits and demonstrating the efficacy of structured knowledge injection.

Abstract

Large vision-language models have improved at describing visual content, but accurate descriptions do not ensure interpretation when meaning depends on knowledge beyond the pixels. Memes expose this gap because they rely on cultural entities, background knowledge, and community conventions. Most meme benchmarks reduce interpretation to labels or holistic scores, obscuring where an explanation breaks down. We introduce MemeBench, a diagnostic benchmark of 1,253 Chinese and English memes with human-written references and quality-controlled VIKR annotations, centered on anime, comics, games, and adjacent online subcultures. Its VIKR schema decomposes explanations into Visual clues, Identity links, Knowledge units, and Reasoning mechanisms. Across 26 LVLMs, every model covers visible content more reliably than the knowledge needed to interpret it, and even the strongest retains a 22.6% Visual-Knowledge gap. To test whether this diagnosis can guide improvement, we introduce KAR, an entity-guided retrieval baseline built on CultureBase. Across four controlled models, KAR raises VIKR Success by 3.6-7.4% and, compared with generic retrieval, repairs more answers and breaks fewer. Yet both retrieval conditions improve Identity and Knowledge while reducing Visual coverage in every comparison. MemeBench reveals whether an interpretation succeeds, what is missing, and whether targeted evidence fills the diagnosed gap.

Problem

• memes
• vision-language models
• cultural knowledge
• interpretation gap
• benchmark

Innovation

• MemeBench
• VIKR schema
• culture-dependent memes
• knowledge-aware retrieval
• large vision-language models