StemFX: Learning Mixing Style Representations via Autoregressive FX Chain Prediction on Source-Separated Stems

2026-07-17

AI Summary

This work addresses the limitations of existing mixing style modeling approaches, which are constrained by fixed numbers of audio effects, reliance on differentiable implementations, or scarcity of multitrack data, making them ill-suited for variable-length, source-dependent effect chains (FX chains). The authors propose an autoregressive modeling framework that predicts FX chains of arbitrary length and type from source-separated pseudo-dry signals. Their method employs a band-split CNN encoder with FiLM conditioning to extract spectral features from dry signals and a Transformer decoder to generate tokenized FX sequences. Crucially, it does not require differentiable effects and is trained on a large-scale dataset comprising 85 effects from the MultiAFx toolkit and 105K songs. Experiments demonstrate significant improvements over baselines in mixing style retrieval, state-of-the-art spectral fidelity and highest subjective preference in style transfer, and over 4,000× faster inference compared to iterative optimization methods.

Abstract

Audio mixing style encompasses the artistic and technical decisions a mix engineer makes, including level balancing, spatialization, and the choice, ordering, and parameterization of audio effects (FX) on each stem. FX chains are a key determinant of this style, yet existing approaches to modeling them remain limited. Some operate on stereo mixtures without explicit per-stem FX chain modeling, others fix the number or type of effects per track, and many require differentiable effect implementations or scarce multitrack datasets. We present StemFX, a framework that learns mixing style representations by autoregressively predicting variable-length FX chains on source-separated stems. A Transformer decoder predicts tokenized FX chains autoregressively, while a band-split multi-band CNN encoder with FiLM conditioning captures per-stem spectral structure. To enable large-scale paired training, we extract pseudo-stems from about 105K songs via source separation and augment them using MultiAFx, a toolkit unifying 85 audio effects from 7 Python libraries. Evaluated on mixing style retrieval, StemFX outperforms all baseline models across all tested chain lengths. On paired mixing style transfer, StemFX achieves the best spectral fidelity and the highest listener preference, over 4000 times faster than iterative optimization.

Problem

• audio mixing style
• FX chains
• source-separated stems
• style representation
• autoregressive prediction

Innovation

• autoregressive FX chain prediction
• mixing style representation
• source-separated stems
• Transformer decoder
• band-split CNN with FiLM