Sample-Constrained Black Box Optimization for Audio Personalization

2025-07-17

AI Summary

This paper addresses black-box audio personalization under finite-sample constraints: optimizing a linear filter (h^*) to maximize user experience across arbitrary audio inputs, given an unknown user satisfaction function and minimal auditory feedback queries. We propose a hybrid-feedback Bayesian optimization framework that jointly leverages global sample-level ratings and fine-grained parameter-level preference comparisons—overcoming the limitation of conventional single-point query strategies. To model heterogeneous feedback sources efficiently, we employ sparse Gaussian process regression. Extensive simulations and real-user experiments demonstrate that our method converges to high-performance filters within approximately 20 interactions. It significantly improves subjective satisfaction in both music and speech scenarios ((p < 0.01)) and achieves a 2.3× improvement in query efficiency over baseline methods.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Applications of semantic technologies for improving search, browsing, recommendation, personalization

Abstract

We consider the problem of personalizing audio to maximize user experience. Briefly, we aim to find a filter $h^*$, which applied to any music or speech, will maximize the user's satisfaction. This is a black-box optimization problem since the user's satisfaction function is unknown. Substantive work has been done on this topic where the key idea is to play audio samples to the user, each shaped by a different filter $h_i$, and query the user for their satisfaction scores $f(h_i)$. A family of ``surrogate" functions is then designed to fit these scores and the optimization method gradually refines these functions to arrive at the filter $hat{h}^*$ that maximizes satisfaction. In certain applications, we observe that a second type of querying is possible where users can tell us the individual elements $h^*[j]$ of the optimal filter $h^*$. Consider an analogy from cooking where the goal is to cook a recipe that maximizes user satisfaction. A user can be asked to score various cooked recipes (e.g., tofu fried rice) or to score individual ingredients (say, salt, sugar, rice, chicken, etc.). Given a budget of $B$ queries, where a query can be of either type, our goal is to find the recipe that will maximize this user's satisfaction. Our proposal builds on Sparse Gaussian Process Regression (GPR) and shows how a hybrid approach can outperform any one type of querying. Our results are validated through simulations and real world experiments, where volunteers gave feedback on music/speech audio and were able to achieve high satisfaction levels. We believe this idea of hybrid querying opens new problems in black-box optimization and solutions can benefit other applications beyond audio personalization.

Problem

• Personalizing audio filters to maximize user satisfaction.
• Optimizing hybrid querying for black-box audio customization.
• Balancing recipe and ingredient feedback for optimal results.

Innovation

• Uses hybrid querying for audio personalization
• Employs Sparse Gaussian Process Regression
• Combines user feedback on filters and elements