Technology Category
Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: GenerationPlanning, Routing, and Scheduling: Temporal Planning
Transformer-Based Rhythm Quantization of Performance MIDI Using Beat Annotations
🤖 AI Summary
This study addresses the challenge of rhythmic quantization in free-tempo MIDI performances by proposing a T5-based Transformer model that aligns expressive performances to a canonical score’s rhythmic grid. The approach integrates prior metrical information through a rhythm-quantization-oriented MIDI tokenizer and a beat-alignment preprocessing framework, augmented with data augmentation strategies including transposition, note deletion, and temporal jittering. To the best of our knowledge, this work is the first to apply a T5-style Transformer to rhythmic quantization, enabling cross-time-signature generalization and generating human-readable scores. Evaluated on the ASAP dataset, the model achieves an onset F1 score of 97.3% and a duration accuracy of 83.3%, with further performance gains observed after instrument-specific fine-tuning.
Technology Category
Application Category
📝 Abstract
Rhythm transcription is a key subtask of notation-level Automatic Music Transcription (AMT). While deep learning models have been extensively used for detecting the metrical grid in audio and MIDI performances, beat-based rhythm quantization remains largely unexplored. In this work, we introduce a novel deep learning approach for quantizing MIDI performances using a priori beat information. Our method leverages the transformer architecture to effectively process synchronized score and performance data for training a quantization model. Key components of our approach include dataset preparation, a beat-based pre-quantization method to align performance and score times within a unified framework, and a MIDI tokenizer tailored for this task. We adapt a transformer model based on the T5 architecture to meet the specific requirements of rhythm quantization. The model is evaluated using a set of score-level metrics designed for objective assessment of quantization performance. Through systematic evaluation, we optimize both data representation and model architecture. Additionally, we apply performance and score augmentations, such as transposition, note deletion, and performance-side time jitter, to enhance the model's robustness. Finally, a qualitative analysis compares our model's quantization performance against state-of-the-art probabilistic and deep-learning models on various example pieces. Our model achieves an onset F1-score of 97.3% and a note value accuracy of 83.3% on the ASAP dataset. It generalizes well across time signatures, including those not seen during training, and produces readable score output. Fine-tuning on instrument-specific datasets further improves performance by capturing characteristic rhythmic and melodic patterns. This work contributes a robust and flexible framework for beat-based MIDI quantization using transformer models.
Problem
Research questions and friction points this paper is trying to address.
rhythm quantization
MIDI performance
beat annotation
automatic music transcription
metrical grid
Innovation
Methods, ideas, or system contributions that make the work stand out.
Transformer
rhythm quantization
MIDI transcription
beat annotation
data augmentation
🔎 Similar Papers
No similar papers found.
