Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

2026-07-21

AI Summary

This work addresses the challenges in understanding and generating multimodal humor—such as memes and comics—which relies heavily on non-literal semantics, cultural common ground, and communicative intent, far exceeding mere scene description. The authors propose a capability-centered hierarchical framework that systematically encompasses humor recognition, explanatory reasoning, and controllable generation, establishing a unified benchmark, evaluation protocol, and modeling paradigm. By integrating multimodal large language models, alignment techniques, evidence-driven reasoning, and controllable generation strategies, the study clarifies the evolutionary trajectory from specialized fusion architectures toward general-purpose foundation models. It further identifies key bottlenecks: shortcut-prone evaluation metrics, limited coverage of cultural narratives, insufficient evidential support in reasoning, and unresolved safety and copyright risks, thereby charting critical directions for future research.

Abstract

Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, shared cultural knowledge, and communicative intent rather than literal scene description. This survey focuses on visual humor understanding in single-image and multi-panel artifacts, while treating humor generation as an emerging downstream frontier. We position the literature against prior humor, sarcasm, and general MLLM surveys and organize it using a capability-centric hierarchy spanning recognition, interpretation and reasoning, and generation. Under this lens, we synthesize benchmark design, evaluation protocols, and modeling paradigms, tracing the field's shift from task-specific fusion models to large-model approaches based on multimodal alignment, evidence-grounded reasoning, and controlled generation. We conclude by highlighting the main barriers to progress: shortcut-prone evaluation, limited cultural and narrative coverage, weak evidence grounding, and unresolved safety and ownership concerns.

Problem

• multimodal humor
• computational humor
• visual understanding
• cultural knowledge
• non-literal meaning

Innovation

• multimodal LLMs
• computational humor
• evidence-grounded reasoning
• controlled generation
• capability-centric hierarchy