Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretation

2024-04-10

arXiv.org

AI Summary

Metaphor understanding is a critical capability for language models, yet cross-lingual metaphor research has long been hindered by the absence of parallel annotated corpora. To address this, we introduce Meta4XNLI—the first Spanish–English parallel metaphor corpus—built upon the XNLI framework and featuring fine-grained, bilingual-aligned metaphor annotations. It supports cross-lingual metaphor detection, interpretation, and transferability analysis. Meta4XNLI constitutes the first systematic, contrastive resource for multilingual metaphor phenomena and serves as a benchmark for evaluating mainstream multilingual models on metaphor tasks. Experimental results reveal a substantial performance drop (−12.3% average) in metaphor identification across models, with limited cross-lingual transfer efficacy—highlighting the strong language specificity of metaphor. This work establishes a novel evaluation benchmark and analytical toolkit for multilingual semantic understanding.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Cognitive Modeling & Cognitive Systems: Analogy
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Metaphors, although occasionally unperceived, are ubiquitous in our everyday language. Thus, it is crucial for Language Models to be able to grasp the underlying meaning of this kind of figurative language. In this work, we present Meta4XNLI, a novel parallel dataset for the tasks of metaphor detection and interpretation that contains metaphor annotations in both Spanish and English. We investigate language models' metaphor identification and understanding abilities through a series of monolingual and cross-lingual experiments by leveraging our proposed corpus. In order to comprehend how these non-literal expressions affect models' performance, we look over the results and perform an error analysis. Additionally, parallel data offers many potential opportunities to investigate metaphor transferability between these languages and the impact of translation on the development of multilingual annotated resources.

Problem

• Detecting metaphors in multilingual text data
• Interpreting figurative language in English and Spanish
• Assessing cross-lingual metaphor transferability in language models

Innovation

• Parallel dataset for metaphor detection and interpretation
• Monolingual and cross-lingual experiments with language models
• Investigates metaphor transferability and translation impact