CrowdioSet and PaRIRset: Two Datasets Towards Live Music Source Separation

2026-07-30

AI Summary

This work addresses the limited generalization of existing music source separation models to real-world live recordings, which stems from their predominant training on ideal studio data that disregards venue acoustics, sound reinforcement systems, and audience noise. To bridge this gap, the study introduces two novel datasets tailored for live music scenarios: CrowdioSet, which leverages zero-shot vocal conversion to generate realistic synthetic choir performances and integrates authentic environmental noise for audio denoising; and PaRIRset, comprising stereo room impulse responses captured from 40 professional performance venues to enable accurate acoustic simulation. By integrating microphone array recordings with deep learning techniques, the proposed framework significantly outperforms current baselines in both objective metrics and subjective listening quality, demonstrating the efficacy of the new datasets in enhancing source separation performance for live music.

Abstract

Most Music Source Separation (MSS) models do not generalize well to live music recordings because they are trained on studio recordings alone, disregarding the venue acoustics, the speaker system's response and audience noise. We propose to bridge this gap by providing and training a model on two novel datasets. First, we present CrowdioSet: a noise dataset comprising 4800 real ambience tracks from Freesound and synthetic sing-alongs for the vocals in MUSDB18 and MOISESDB datasets, generated from zero-shot singing voice conversions. CrowdioSet enables effective audio denoising for live recordings, resulting in superior separation both in objective and subjective evaluations. Second, we introduce PaRIRset, a stereo impulse response dataset captured across 40 professional concert venues using a microphone array. Our results show that adding PaRIRset RIRs increases the performance of a MSS model compared to using real RIRs from Speech Enhancement tasks alone. We make the examples, code, model weights, PaRIRset, and CrowdioSet freely available to the public.

Problem

• Music Source Separation
• live music recordings
• venue acoustics
• audience noise
• impulse response

Innovation

• Music Source Separation
• Live Music
• CrowdioSet
• PaRIRset
• Room Impulse Response