M$^3$R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

2026-08-06

AI Summary

This work addresses the limitation of existing metaphor understanding benchmarks, which predominantly rely on isolated subtasks and lack evaluation of cross-modal target–source mappings grounded in joint visual and textual evidence. To bridge this gap, we introduce M³R-Bench, a unified multimodal benchmark grounded in Conceptual Metaphor Theory, comprising 1,000 human-verified image–text samples annotated across four layers: metaphor existence, mapping relations, sentiment polarity, and stepwise explanations. We further propose a novel three-stage evaluation framework—evidence identification, mapping construction, and sentiment inference—that reveals current models’ overreliance on textual cues and neglect of visual evidence. Building upon this, we develop M³R-Reasoner, which integrates curriculum-based reasoning supervision with task-aware reinforcement learning to guide multimodal large language models toward evidence–mapping consistent reasoning. Despite using only an 8B-parameter backbone, our model surpasses larger closed-source counterparts across all four metrics, outscoring GPT-5.5 by 28.45 and 30.11 points in visual evidence and sentiment plausibility, respectively, and exceeding Claude-Sonnet-4.6 by an average of 8.00 points.

Abstract

Metaphor enables the understanding of abstract concepts through cross-domain mappings while conveying affective attitudes. In multimodal scenarios, visual and textual information jointly construct Target--Source mappings, requiring both conceptual understanding and cross-modal reasoning. However, existing benchmarks mainly evaluate metaphor understanding through isolated subtasks and lack evidence-grounded explanations, making it difficult to assess whether models establish mappings grounded in visual and textual cues.To address these limitations, we introduce M$^3$R-Bench, a unified and evidence-grounded benchmark containing 1,000 image--text instances with human-verified annotations. Guided by Conceptual Metaphor Theory and theories of nonliteral language understanding, M$^3$R-Bench provides joint annotations for metaphor occurrence, Target--Source mapping, sentiment, and stage-wise explanations following ``evidence identification--mapping establishment--sentiment inference.''Evaluations on M$^3$R-Bench reveal that existing models often overlook visual evidence, rely on superficial textual cues, and produce inaccurate Target--Source mappings, exposing a cross-modal evidence--mapping mismatch. To address this mismatch, we propose M$^3$R-Reasoner, which combines curriculum-based reasoning supervision with task-aware reinforcement learning to align model reasoning with metaphor interpretation. Experiments show that, with only an 8B-parameter backbone, M$^3$R-Reasoner outperforms larger proprietary MLLMs across four unified-task metrics and improves Visual Evidence and Sentiment Justification scores over GPT-5.5 by 28.45 and 30.11 points, respectively, while surpassing Claude-Sonnet-4.6 by 8.00 points in mean rubric score. The dataset and code are available at https://github.com/hongshi4/M3R-Bench.

Problem

• multimodal metaphor understanding
• evidence-grounded benchmark
• Target--Source mapping
• cross-modal reasoning
• conceptual metaphor

Innovation

• evidence-grounded reasoning
• multimodal metaphor understanding
• Target--Source mapping
• curriculum-based reasoning
• task-aware reinforcement learning