ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models

2026-08-04

AI Summary

This study addresses the neglect of cultural differences in existing visual emotion understanding approaches and the absence of reliable, balanced cross-cultural annotated data. To bridge this gap, we introduce the task of culture-conditioned visual emotion understanding and present the first benchmark dataset comprising 6,792 artworks annotated with emotion labels and explanations from Chinese, English, and Arabic cultural perspectives. Using this dataset, we evaluate the zero-shot performance of 16 state-of-the-art multimodal large language models, revealing that all achieve accuracy below 50%. To improve performance, we propose a training-free retrieval-augmented framework that injects explicit cultural knowledge into the models via a concept-based cultural emotion knowledge base, significantly enhancing both emotion prediction accuracy and the interpretability of generated explanations.

Abstract

Existing visual emotion understanding methods typically ignore cultural variations in emotional perception. We introduce culture-conditioned visual emotion understanding, a task that predicts the culture-specific emotional perception of a given image and explains the underlying rationale. Although related benchmarks exist, they are limited by inconsistent individual annotations, which hinder the derivation of majority-supported culture-level emotion labels, and imbalanced cultural coverage. Thus, we present ArtECulture, a benchmark containing 6,792 artworks with culture-specific emotion labels and explanations across English, Chinese, and Arabic cultures, with balanced Western and non-Western content. Evaluations of 16 open- and closed-source Multimodal Large Language Models (MLLMs) under a zero-shot setting reveal that the task remains challenging, with the best model achieving below 50\% accuracy. To address this limitation, we introduce a retrieval-augmented culture-conditioned emotion understanding framework, which leverages a concept-based cultural emotion knowledge base to inject explicit cultural knowledge into MLLMs without additional training. The framework improves both culturally aligned emotion prediction and grounded explanation generation. Our benchmark and code will be publicly released.

Problem

• visual emotion understanding
• cultural variation
• multimodal large language models
• culture-conditioned emotion
• benchmark

Innovation

• culture-conditioned emotion understanding
• multimodal large language models
• retrieval-augmented framework
• cultural knowledge injection
• visual emotion benchmark