Versatile Symbolic Music-for-Music Modeling via Function Alignment

2025-06-18

AI Summary

This paper proposes a label-free, unified modeling paradigm for symbolic music, framing diverse tasks—including chord recognition, melody generation, and drum pattern generation—as a single “music-to-music” sequence modeling problem. Methodologically, it introduces (1) a novel *function alignment* paradigm that implicitly encodes functional semantics (e.g., harmonic role, rhythmic function) via intrinsic structural patterns in note sequences, eliminating reliance on external annotations; (2) a lightweight, learnable adapter that bridges two pre-trained language model encoders, enabling parameter-efficient, task-agnostic modeling; and (3) a symbolic-music-specific tokenization scheme integrated with functional semantic alignment. The framework achieves state-of-the-art performance across multiple symbolic music understanding and generation benchmarks. All code, pretrained models, and interactive demos are publicly released.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Many music AI models learn a map between music content and human-defined labels. However, many annotations, such as chords, can be naturally expressed within the music modality itself, e.g., as sequences of symbolic notes. This observation enables both understanding tasks (e.g., chord recognition) and conditional generation tasks (e.g., chord-conditioned melody generation) to be unified under a music-for-music sequence modeling paradigm. In this work, we propose parameter-efficient solutions for a variety of symbolic music-for-music tasks. The high-level idea is that (1) we utilize a pretrained Language Model (LM) for both the reference and the target sequence and (2) we link these two LMs via a lightweight adapter. Experiments show that our method achieves superior performance among different tasks such as chord recognition, melody generation, and drum track generation. All demos, code and model weights are publicly available.

Problem

• Unify music understanding and generation tasks via music-for-music modeling
• Enable parameter-efficient solutions for symbolic music tasks
• Link pretrained language models with lightweight adapters for versatility

Innovation

• Uses pretrained Language Model for sequences
• Links LMs via lightweight adapter
• Unifies music tasks under one paradigm