Separate-and-Detect: Unified Drum Transcription and Stem Generation via Latent Diffusion

2026-08-02

AI Summary

This work addresses the limitation of conventional drum transcription methods, which output only symbolic events and discard acoustically rich stem information essential for editing and mixing. To overcome this, the authors propose a unified “separation–detection” framework that integrates transcription and source separation within an interpretable generative front-end. The approach first employs a five-track drum sound diffusion model operating in a VAE latent space to perform source separation, followed by a fixed onset detector to generate symbolic events. During training, auxiliary onset and timbre branches (OB and TB) are introduced to jointly optimize separation and detection performance. Experiments demonstrate that the method surpasses U-Net baselines in transcription F1 scores on the MDB Drums and ENST-Drums datasets, outperforms existing end-to-end systems on kick and snare drums, and simultaneously produces high-quality audio stems.

Abstract

Automatic Drum Transcription (ADT) is commonly formulated as a direct mapping from a music mixture to symbolic drum events. While effective for transcription, this formulation discards the acoustic stems that are useful for editing, remixing, and production. We revisit an alternative separate-and-detect formulation, where a drum source separation front end first produces five editable drum stems, and a fixed onset detector then converts each stem into symbolic events. The separator is built on a five-stem latent diffusion model that jointly generates kick, snare, toms, hi-hats, and cymbals in a compact VAE latent space. We further study two training-only auxiliary branches--an onset branch (OB) and a timbre branch (TB)--which shape the separator during learning but are discarded at inference. Trained on synthetic drum multitracks and evaluated on MDB Drums and ENST-Drums, the proposed pipeline consistently improves over a strong U-Net-based drum separation baseline in overall transcription F1. It also outperforms a representative end-to-end ADT system on kick and snare F1 under our evaluation protocol, while additionally providing separated audio stems. The ablation results show that OB gives the most stable transcription gains, whereas TB changes the trade-off between reconstruction, acoustic stem quality, and onset detection. These results suggest that generative drum demixing can serve not only as a source separation model, but also as a practical front end for interpretable drum transcription.

Problem

• Automatic Drum Transcription
• Drum Source Separation
• Audio Stems
• Symbolic Drum Events
• Music Production

Innovation

• latent diffusion
• drum transcription
• source separation
• onset detection
• VAE latent space