Innovation
Methods, ideas, or system contributions that make the work stand out.
Music Evaluation
Model-Intrinsic Signals
Sparse Autoencoder
MusicGen
Predictive Entropy
Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals
🤖 AI Summary
This study investigates whether music generation models can perceive the quality of their own outputs and whether such perception aligns with human evaluation. Specifically, this work extracts hidden representations and predictive signals—including loss, entropy, and Sparse Autoencoder (SAE) concepts—from MusicGen to train lightweight regression models that estimate subjective music ratings. To our knowledge, this is the first research to demonstrate a strong correlation between intrinsic model signals and human assessments, proposing a novel perceptual quality prediction mechanism grounded in the SAE latent space. Extensive experiments across five benchmarks reveal that SAE latent features encapsulate the core predictive signals, effectively enabling automated music quality assessment.
📝 Abstract
Current music generative models can produce high-quality music, but does this ability imply that they ``understand'' the musical qualities of their outputs, and is that understanding aligned with human evaluation? Previous attempts to use the likelihood of a generative model to evaluate music, an approach commonly used in text, have proven unsuccessful, leading researchers to rely on standalone supervised music evaluation models. In this paper, we answer this question affirmatively: we show that a model's intrinsic signals---derived from its hidden representations and predictions---are strongly correlated with human ratings. In particular, we study MusicGen and consider three types of features: (1) prediction loss, (2) prediction entropy, and (3) concepts extracted from the model using a sparse autoencoder (SAE). Using these features, we train a lightweight prediction model to estimate subjective ratings. We evaluate these features both individually and in combination. We hypothesize that these signals parallel the listening process: the temporal and frequency-domain structure of loss and entropy reflects listeners' expectation and surprise, while gradient directions in SAE latent space predict perceived quality. Experiments on five human-evaluation benchmarks spanning continuous ratings and pairwise preferences confirm this hypothesis, with SAE latents carrying most of the predictive signal.
🔎 Similar Papers
No similar papers found.
