PAGURI: a user experience study of creative interaction with text-to-music models

2024-07-05

Electronics

AI Summary

This study addresses the practical deployment challenges of text-to-music (TTM) models in professional music practice—the first systematic user experience investigation targeting practicing musicians. We developed an online interactive prototype enabling music generation and personalized fine-tuning, and conducted semi-structured interviews coupled with thematic analysis to examine TTM’s usability, limitations, and integration pathways in authentic creative workflows. We propose the user-centered PAGURI research framework and empirically identify three core bottlenecks: (1) semantic ambiguity in textual prompts, (2) insufficient granularity of musical control, and (3) deep misalignment with established production workflows. Results confirm TTM’s substantial value for creative inspiration and personalization potential. The findings yield empirically grounded design principles and concrete improvement directions for next-generation TTM systems—emphasizing trustworthiness, controllability, and seamless workflow integration.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

In recent years, text-to-music models have been the biggest breakthrough in automatic music generation. While they are unquestionably a showcase of technological progress, it is not clear yet how they can be realistically integrated into the artistic practice of musicians and music practitioners. This paper aims to address this question via Prompt Audio Generation User Research Investigation (PAGURI), a user experience study where we leverage recent text-to-music developments to study how musicians and practitioners interact with these systems, evaluating their satisfaction levels. We developed an online tool through which users can generate music samples and/or apply recently proposed personalization techniques based on fine-tuning to allow the text-to-music model to generate sounds closer to their needs and preferences. Using semi-structured interviews, we analyzed different aspects related to how participants interacted with the proposed tool to understand the current effectiveness and limitations of text-to-music models in enhancing users’ creativity. Our research centers on user experiences to uncover insights that can guide the future development of TTM models and their role in AI-driven music creation. Additionally, they offered insightful perspectives on potential system improvements and their integration into their music practices. The results obtained through the study reveal the pros and cons of the use of TTMs for creative endeavors. Participants recognized the system’s creative potential and appreciated the usefulness of its personalization features. However, they also identified several challenges that must be addressed before TTMs are ready for real-world music creation, particularly issues of prompt ambiguity, limited controllability, and integration into existing workflows.

Problem

• Studying musician interaction with text-to-music generation systems
• Evaluating user satisfaction and creative enhancement limitations
• Identifying challenges in real-world music creation integration

Innovation

• Online tool for text-to-music sample generation
• Personalization via fine-tuning for user preferences
• Semi-structured interviews to evaluate user experience