Music and art: a study in cross-modal interpretation

2025-01-09

AI Summary

This study investigates how music cross-modally enhances art viewing experiences. Addressing limitations of emotion-centric paradigms, it employs a mixed-methods experiment with 138 participants, integrating quantitative surveys, open-ended textual responses subjected to thematic coding, and computational sentiment analysis. Results systematically demonstrate that thematic congruence and narrative coherence—rather than emotional valence or arousal matching—are the primary determinants of effective audiovisual integration; musical tempo exerts only marginal influence. The study thus shifts theoretical emphasis from affective alignment to semantic and structural coupling between auditory and visual modalities. Six evidence-based, actionable guidelines for music–art curatorial integration are proposed. Findings confirm that contextually aligned music significantly improves viewers’ conceptual understanding of artworks and aesthetic appreciation. These outcomes provide empirical validation and methodological frameworks for museum exhibition design, digital curation, and multisensory art practice.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Our study has investigated the effect of music on the experience of viewing art, investigating the factors which create a sense of connectivity between the two forms. We worked with 138 participants, and included multiple choice and open-ended questions. For the latter, we performed both a qualitative analysis and also sentiment analysis using text-mining. We investigated the relationship between the user experience and the emotions in the artwork and music. We found that, besides emotion, theme, story, and to a lesser extent music tempo were factors which helped form connections between artwork and music. Overall, participants rated the music as being helpful in developing an appreciation of the art. We propose guidelines for using music to enhance the experience of viewing art, and we propose directions for future research.

Problem

• Music Perception
• Art Appreciation
• Multimodal Experience

Innovation

• Music-Art Interaction
• Perceptual Enhancement
• Matching Parameters