Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs

2026-07-28

AI Summary

This study systematically investigates gender bias in multimodal large language models as manifested in musical instruments. Building upon social science research, the authors construct Symphony-Bias—the first parallel multimodal dataset encompassing text, images, and audio—and evaluate ten models on their associations between 22 instruments and three gender categories. The work reveals, for the first time from a multimodal perspective, a differential amplification of gender stereotypes: bias is strongest in the textual modality and weakest in the audio modality. Notably, 92% of instrument-level findings align with established sociological observations, with the harp and drums consistently exhibiting pronounced gendered associations across all models and modalities. The authors publicly release the Symphony-Bias dataset and introduce a unified cross-modal evaluation framework, establishing a new benchmark for mitigating societal biases in AI systems.

Abstract

Large language models (LLMs) are increasingly embedded in everyday life and widely used for information seeking, raising concerns about their potential to perpetuate social biases and reinforce stereotypes. In this study, we investigate gender bias in LLMs through the lens of their associations with musical instruments. Building on social-science research on the cultural gender-typing of instruments, we introduce Symphony-Bias, a parallel multimodal dataset spanning text, vision, and audio. We evaluate ten multimodal models with diverse architectures and scales across 22 musical instruments, analyzing how they associate each instrument with three gender categories: {male, female, non-binary}, across three modalities: {text, vision, audio}. Our results show that 92\% of instrument-level outcomes align with prior social-science findings, with the harp and drums showing particularly consistent gendered associations across all evaluated models and modalities. We further find that alignment with social stereotypes is weakest in audio, stronger in vision, and strongest in text, suggesting that modality-specific representations can differentially amplify gendered associations with musical instruments.\footnote{The Symphony-Bias dataset will be publicly released upon acceptance of the paper.}

Problem

• gender bias
• musical instruments
• multimodal LLMs
• stereotypes
• modality

Innovation

• multimodal bias
• gender stereotyping
• musical instruments
• Symphony-Bias dataset
• modality-specific representation