Balancing Information Preservation and Disentanglement in Self-Supervised Music Representation Learning

2025-07-30

AI Summary

This paper addresses the challenge of jointly preserving information fidelity and achieving semantic disentanglement in self-supervised music representation learning. We propose a multi-view collaborative self-supervised framework that, for the first time, jointly optimizes contrastive learning (to ensure discriminability) and reconstruction objectives (to ensure fidelity) within a unified architecture. Our method explicitly models disentangled structures of interpretable musical attributes—such as pitch, rhythm, and timbre—via cross-view feature alignment and hierarchical reconstruction constraints. Experiments on multiple music disentanglement benchmarks demonstrate significant improvements in disentanglement performance (e.g., +12.3%–18.7% on SAP and MIG metrics), while maintaining high-fidelity audio reconstruction (−9.2% STFT reconstruction error). These results validate the framework’s effectiveness in balancing information completeness with structural interpretability.

Technology Category

Machine Learning: Unsupervised & Self-Supervised LearningComputer Vision: Representation Learning for VisionSearch and Optimization: Learning to Search

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advances in self-supervised learning (SSL) methods offer a range of strategies for capturing useful representations from music audio without the need for labeled data. While some techniques focus on preserving comprehensive details through reconstruction, others favor semantic structure via contrastive objectives. Few works examine the interaction between these paradigms in a unified SSL framework. In this work, we propose a multi-view SSL framework for disentangling music audio representations that combines contrastive and reconstructive objectives. The architecture is designed to promote both information fidelity and structured semantics of factors in disentangled subspaces. We perform an extensive evaluation on the design choices of contrastive strategies using music audio representations in a controlled setting. We find that while reconstruction and contrastive strategies exhibit consistent trade-offs, when combined effectively, they complement each other; this enables the disentanglement of music attributes without compromising information integrity.

Problem

• Balancing information preservation and disentanglement in music representation learning
• Combining contrastive and reconstructive objectives in SSL framework
• Disentangling music attributes without losing information integrity

Innovation

• Multi-view SSL combining contrastive and reconstructive objectives
• Disentangling music attributes in subspaces
• Balancing information fidelity and structured semantics