Dolph2Vec: Self-Supervised Representations of Dolphin Vocalizations

2026-06-10

AI Summary

This work addresses the limitations of existing self-supervised models in bioacoustics, which prioritize cross-species generalization and fail to capture the fine-grained structure of dolphin communication systems. The authors propose Dolph2Vec—the first self-supervised representation model tailored specifically for dolphin vocalizations—adapting and refining the Wav2Vec2.0 architecture to learn nuanced acoustic features of whistles from five years of longitudinal audio recordings. Dolph2Vec establishes the first large-scale, species-specific self-supervised framework for dolphin vocalizations, significantly outperforming general-purpose baselines in signature whistle classification and whistle detection tasks. Moreover, its learned embedding space exhibits an interpretable structure aligned with established bioacoustic categories, offering both strong technical performance and meaningful scientific insights into dolphin communication.

Abstract

Self-supervised learning (SSL) has opened new opportunities in bioacoustics by enabling scalable modeling of animal vocalizations without the need for expensive manual annotation. However, current SSL models in this domain prioritize broad generalization across species and are not optimized for uncovering the fine-grained structure of individual communication systems. In this work, we collect and release a novel dataset of over five years of longitudinal recordings, from five known dolphins in a semi-naturalistic marine environment, an unprecedented resource for studying dolphin communication. We adapt the Wav2Vec2.0 Baevski et al. (2020) architecture to this domain and introduce Dolph2Vec, the first large-scale, species-specific SSL model trained exclusively on this data. We benchmark our model on two biologically relevant tasks: signature whistle classification and whistle detection. Dolph2Vec significantly outperforms general-purpose baselines in both tasks. Beyond performance, we show that learned embeddings and codebook structure capture interpretable acoustic units aligned with dolphin whistle categories and possibly sub-whistle structure, enabling fine-grained analysis of communication patterns. Our findings demonstrate how SSL can serve as both a model and a scientific tool to explore hypotheses in animal communication research.

Problem

• dolphin vocalizations
• self-supervised learning
• fine-grained structure
• animal communication
• bioacoustics

Innovation

• self-supervised learning
• dolphin vocalizations
• species-specific model
• acoustic embeddings
• signature whistle