Expressive Music Data Processing and Generation

2025-03-14

AI Summary

To address the lack of expressivity and structural coherence in AI-generated music, this paper proposes an auditory-perception-driven generative framework. First, Weber’s Law is introduced into music data preprocessing to faithfully model subtle performance variations. Second, a conditional dependency model over multidimensional musical parameters is constructed, enabling joint modeling of multiple output sequences via the probabilistic chain rule. Third, a stepwise conditional sampling strategy coupled with entropy-guided filtering is designed, integrating information-theoretic aesthetics to quantify listener pleasure and information gain. Experiments demonstrate significant improvements over baselines in expressivity fidelity and phrase-level coherence, as validated by MuseScore metrics, Earth Mover’s Distance (EMD), and subjective evaluations. Moreover, this work establishes the first interpretable information-aesthetic evaluation framework for AI music generation.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Musical expressivity and coherence are indispensable in music composition and performance, while often neglected in modern AI generative models. In this work, we introduce a listening-based data-processing technique that captures the expressivity in musical performance. This technique derived from Weber's law reflects the human perceptual truth of listening and preserves musical subtlety and expressivity in the training input. To facilitate musical coherence, we model the output interdependencies among multiple arguments in the music data such as pitch, duration, velocity, etc. in the neural networks based on the probabilistic chain rule. In practice, we decompose the multi-output sequential model into single-output submodels and condition previously sampled outputs on the subsequent submodels to induce conditional distributions. Finally, to select eligible sequences from all generations, a tentative measure based on the output entropy was proposed. The entropy sequence is set as a criterion to select predictable and stable generations, which is further studied under the context of informational aesthetic measures to quantify musical pleasure and information gain along the music tendency.

Problem

• Captures musical expressivity using listening-based data-processing.
• Models output interdependencies for musical coherence in neural networks.
• Proposes entropy-based measure to select predictable and stable music sequences.

Innovation

• Listening-based data-processing captures musical expressivity.
• Probabilistic chain rule models output interdependencies in music.
• Output entropy measures select predictable, stable music sequences.