Technology Category
Machine Learning: Deep Generative Models & AutoencodersPlanning, Routing, and Scheduling: Mixed Discrete/Continuous PlanningSearch and Optimization: Mixed Discrete/Continuous Search
WildFX: A DAW-Powered Pipeline for In-the-Wild Audio FX Graph Modeling
🤖 AI Summary
AI-driven audio effect modeling faces a fundamental challenge: existing neural black-box approaches fail to accurately replicate the complex signal routing, parameter coupling, and authentic behavior of professional DSP workflows. Method: This paper introduces the first industrial-grade automated data generation framework for digital audio workstations (DAWs), supporting seamless integration of VST/VST3/LV2/CLAP plugins—including advanced features such as sidechaining and frequency splitting—and enabling efficient configuration via a lightweight metadata interface. The framework employs Docker-based containerization, combined with differentiable signal-flow graph simulation and black-box parameter estimation, to achieve hybrid graph blind identification. Contribution/Results: Experiments demonstrate that, under equivalent computational budgets, our method significantly outperforms state-of-the-art differentiable plugin approaches in both effect graph topology recovery and parameter estimation accuracy. The open-source implementation establishes a new paradigm bridging neural audio modeling and real-world DSP practice.
Technology Category
Application Category
📝 Abstract
Despite rapid progress in end-to-end AI music generation, AI-driven modeling of professional Digital Signal Processing (DSP) workflows remains challenging. In particular, while there is growing interest in neural black-box modeling of audio effect graphs (e.g. reverb, compression, equalization), AI-based approaches struggle to replicate the nuanced signal flow and parameter interactions used in professional workflows. Existing differentiable plugin approaches often diverge from real-world tools, exhibiting inferior performance relative to simplified neural controllers under equivalent computational constraints. We introduce WildFX, a pipeline containerized with Docker for generating multi-track audio mixing datasets with rich effect graphs, powered by a professional Digital Audio Workstation (DAW) backend. WildFX supports seamless integration of cross-platform commercial plugins or any plugins in the wild, in VST/VST3/LV2/CLAP formats, enabling structural complexity (e.g., sidechains, crossovers) and achieving efficient parallelized processing. A minimalist metadata interface simplifies project/plugin configuration. Experiments demonstrate the pipeline's validity through blind estimation of mixing graphs, plugin/gain parameters, and its ability to bridge AI research with practical DSP demands. The code is available on: https://github.com/IsaacYQH/WildFX.
Problem
Research questions and friction points this paper is trying to address.
AI struggles to model professional DSP workflows accurately
Existing approaches diverge from real-world audio effect tools
Lack of datasets with rich effect graphs for AI training
Innovation
Methods, ideas, or system contributions that make the work stand out.
Docker-containerized pipeline for audio mixing datasets
Seamless integration of cross-platform commercial plugins
Minimalist metadata interface for project configuration
🔎 Similar Papers
No similar papers found.
