🤖 AI Summary
This work addresses the challenge of detecting and localizing target sounds in complex acoustic scenes by proposing a unified encoder-based shared representation learning framework. Departing from conventional conditional embedding mechanisms, the method jointly encodes reference and mixture audio signals within a shared semantic space and employs multi-task learning to simultaneously optimize detection and localization performance. By innovatively aligning audio embeddings and enabling end-to-end training, the approach significantly enhances generalization to unseen sound categories while simplifying model architecture. Evaluated on the URBAN-SED dataset, the proposed method achieves a segment-level F1 score of 83.15% and an overall accuracy of 95.17%, establishing a new state-of-the-art performance.
📝 Abstract
Human listeners exhibit the remarkable ability to segregate a desired sound from complex acoustic scenes through selective auditory attention, motivating the study of Targeted Sound Detection (TSD). The task requires detecting and localizing a target sound in a mixture when a reference audio of that sound is provided. Prior approaches, rely on generating a sound-discriminative conditional embedding vector for the reference and pairing it with a mixture encoder, jointly optimized with a multi-task learning approach. In this work, we propose a unified encoder architecture that processes both the reference and mixture audio within a shared representation space, promoting stronger alignment while reducing architectural complexity. This design choice not only simplifies the overall framework but also enhances generalization to unseen classes. Following the multi-task training paradigm, our method achieves substantial improvements over prior approaches, surpassing existing methods and establishing a new state-of-the-art benchmark for targeted sound detection, with a segment-level F1 score of 83.15% and an overall accuracy of 95.17% on the URBAN-SED dataset.