Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrapping

2026-08-03

AI Summary

Existing symbolic music generation approaches struggle to explicitly model the implicit musical style embedded in audio, often yielding outputs lacking authentic stylistic expression. This work proposes a cross-modal guidance framework that uniquely integrates cross-modal alignment with generative modeling. Leveraging a BLIP-2–inspired architecture, the method employs a Querying Transformer to extract implicit style representations from reference audio via a pretrained audio-language model and uses these representations to guide symbolic music generation. The approach enables joint control by both a melodic lead sheet and reference audio, facilitating high-fidelity, controllable stylized piano arrangements. It achieves significant improvements in style alignment and musical quality across tasks including piano cover generation, style transfer, and audio-to-MIDI retrieval.

Abstract

What is music style? Though often described using text labels such as "swing," "classical," or "emotional," the real style remains implicit and hidden in concrete music examples. In this paper, we introduce a cross-modal framework that learns implicit music styles from raw audio and applies them to symbolic music generation. Inspired by BLIP-2, our model leverages a Querying Transformer (Q-Former) to extract style representations from a large, pre-trained audio language model (LM), and further applies them to condition a symbolic LM for generating piano arrangements. We adopt a two-stage training strategy: contrastive learning to align auditory style with symbolic expression, followed by generative modeling for music arrangement. Our model generates piano performances jointly conditioned on a lead sheet (content) and a reference audio example (style), enabling controllable and stylistically faithful arrangement. Experiments demonstrate the effectiveness of our approach in piano cover generation, style transfer, and audio-to-MIDI retrieval, achieving substantial improvements in style-aware alignment and music quality.

Problem

• music style
• piano arrangement
• cross-modal learning
• symbolic music generation
• style transfer

Innovation

• cross-modal learning
• music style transfer
• Querying Transformer
• symbolic music generation
• audio-to-MIDI alignment