Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts

2026-09-26

AI Summary

This study addresses the limited fine-grained control over melodic motifs in existing symbolic music generation models and the prohibitive cost of training controllable foundation models from scratch. We propose MotiGen, a framework that introduces a novel "post-hoc" controllability mechanism for pretrained models without requiring massive annotated datasets. Specifically, MotiGen reinforces motif correlations by injecting structured motif prompts with scalar attention biases, and performs instruction tuning through a two-stage curriculum learning strategy combining focus-cropped segments and complete scores. Experimental results demonstrate that over 92.3% of the generated compositions successfully incorporate the designated motifs. By circumventing expensive retraining, this approach significantly enhances both the controllability and practical utility of AI-assisted music composition.

Abstract

Deep learning has transformed symbolic music generation by borrowing the training paradigms of large language models, with systems such as NotaGen now producing complete, stylistically convincing classical scores from a short prompt. These systems could become powerful creative partners, helping musicians generate endless possibilities. However, current systems expose almost no control handles on the music itself. In principle, control handles could be built into a foundation model trained from scratch, but this is rarely practical without massive amounts of quality annotated data and compute resources. We therefore present MotiGen, a recipe for retrofitting pretrained symbolic music models to use new instruction prompts. MotiGen injects a musical motif as a structured prompt line, reinforces it with a scalar attention bias toward the motif tokens, and learns the association with a two-phase curriculum. First, it learns from focused excerpts cropped around motif occurrences in the training data, then full scores including the motifs. Our experiments show that our model composes with the prompted motif in over 92.3\% of generated pieces. Generated pieces using a variety of motifs are included in our sample site: https://motigen-site.github.io/.

Problem

• symbolic music generation
• controllable generation
• musical motif
• pretrained models

Innovation

• Controllable Music Generation
• Motif Injection
• Attention Bias
• Curriculum Learning
• Pretrained Model Retrofitting