Large Language Models' Internal Perception of Symbolic Music

2025-07-17

AI Summary

This study investigates whether large language models (LLMs), trained exclusively on text without explicit musical supervision, can implicitly acquire representations of symbolic music structure. Methodologically, we prompt LLMs with plain-text instructions to generate MIDI sequences, construct a symbolic music dataset from these outputs, and train dedicated neural networks for downstream tasks—including music genre classification, style identification, and melody completion. We provide the first empirical evidence that LLMs develop internal representations of core musical concepts—such as pitch, rhythm, and tonality—as demonstrated by the viability of their generated MIDI data for supervised learning: classification accuracy approaches that of fully supervised baselines, and melody completions exhibit musically coherent phrasing and harmonic consistency. Results indicate that LLMs possess nascent—but incomplete—reasoning capabilities over symbolic music structure, establishing a novel paradigm for zero-shot and few-shot music generation and understanding.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Large language models (LLMs) excel at modeling relationships between strings in natural language and have shown promise in extending to other symbolic domains like coding or mathematics. However, the extent to which they implicitly model symbolic music remains underexplored. This paper investigates how LLMs represent musical concepts by generating symbolic music data from textual prompts describing combinations of genres and styles, and evaluating their utility through recognition and generation tasks. We produce a dataset of LLM-generated MIDI files without relying on explicit musical training. We then train neural networks entirely on this LLM-generated MIDI dataset and perform genre and style classification as well as melody completion, benchmarking their performance against established models. Our results demonstrate that LLMs can infer rudimentary musical structures and temporal relationships from text, highlighting both their potential to implicitly encode musical patterns and their limitations due to a lack of explicit musical context, shedding light on their generative capabilities for symbolic music.

Problem

• How LLMs implicitly model symbolic music structures
• Generating symbolic music from text prompts without training
• Evaluating LLMs' musical pattern recognition and generation capabilities

Innovation

• Generates symbolic music from text prompts
• Trains neural networks on LLM-generated MIDI
• Evaluates musical pattern recognition and generation