DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning

2026-08-03

AI Summary

This work addresses the challenges of audio synthesizer inversion—namely, the many-to-one mapping of parameters, perceptual inconsistency, and non-differentiability of synthesizers—by introducing, for the first time, a discrete diffusion mechanism to this task. The problem is formulated as conditional discrete parameter generation, leveraging a masked discrete diffusion model to circumvent the sequential dependencies inherent in autoregressive approaches and the continuous relaxation bias of flow matching. The framework is further refined through end-to-end fine-tuning with GRPO-style reinforcement learning rewards derived from rendered audio. Evaluated on the Dexed dataset, the proposed method matches the performance of existing baselines during supervised training and demonstrates significantly improved out-of-domain audio matching after reward-based fine-tuning.

Abstract

Synthesizer inversion is challenging for two main reasons: 1) Distinct parameter configurations can produce perceptually similar sounds. 2) Parameter-space losses often fail to reflect rendered audio similarity, while the synthesizer being a non-differentiable black box prevents simple audio-domain supervision. To address the one-to-many mapping induced by the first challenge, we formulate synthesizer inversion as conditional generation over discrete synthesizer parameters and use masked discrete diffusion as the generator. This treatment additionally avoids the fixed-order assumption of autoregressive models and the continuous-relaxation mismatch of flow matching when modeling categorical synthesizer controls. To address the second challenge, we further fine-tune the model with GRPO-style audio-domain rewards computed from rendered outputs. Experiments on Dexed show that, after supervised training, the discrete diffusion model is competitive with autoregressive and flow-matching baselines, and reward-based fine-tuning further improves out-of-domain audio matching performance. Code and demos are available at: https://github.com/DDSynth-RL/DDSynthRL.

Problem

• synthesizer inversion
• audio synthesis
• discrete diffusion
• non-differentiable black box
• parameter-audio mapping

Innovation

• discrete diffusion
• synthesizer inversion
• reinforcement learning
• audio-domain reward
• masked generation