A Dual Evaluation for Music Transcription

2026-08-05

AI Summary

Existing automatic music transcription evaluation methods struggle to balance score readability and fidelity to the original performance. This work proposes a dual evaluation framework that assesses transcription quality along two complementary dimensions: symbolic similarity to the reference score and perceptual similarity of synthesized playback. Through a large-scale listening study involving 230 piano recordings and over 100 participants, we provide the first systematic validation of the complementarity between these dimensions and demonstrate that the low-cost metric CLEWS aligns closely with human auditory judgments. Leveraging these insights, we develop Rubato, an end-to-end transcription system that significantly outperforms existing approaches in notational accuracy while maintaining competitive performance in playback similarity.

Abstract

Automatic music transcription systems produce sheet music that can be read and played back. We argue that these two targets call for complementary evaluations of notation similarity to a reference score and playback similarity to the original performance, respectively. Our study considers notation similarity metrics from the optical music recognition literature and a wide range of playback-similarity methods validated through a listening study across over 100 participants and 230 piano recordings covering 23 works, 30 performers, and six composers. We find, fortuitously, that the playback similarity metric that correlates best with human judgments, CLEWS, is also the cheapest to run. We also find that the two evaluation dimensions favor different systems among a collection of 24 pipelines formed by pairing eight audio-to-MIDI models with three MIDI-to-score converters, with the latter component systematically determining the favored objective. The complementarity between metrics also holds when adding to the pool Rubato, a new end-to-end system that offers substantially improved notation similarity while remaining competitive, though not the best, on playback similarity.

Problem

• music transcription
• notation similarity
• playback similarity
• evaluation metrics
• automatic music transcription

Innovation

• dual evaluation
• music transcription
• notation similarity
• playback similarity
• CLEWS