MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models

2026-10-06

AI Summary

This study addresses the limitation of existing conditional injection mechanisms in state space models (SSMs), which struggle to alter underlying temporal dynamics. We propose MaRK, a framework that maps context vectors into bounded modulations of SSM recurrence parameters to achieve dynamic operator conditioning. By introducing Markov parameter sequence modulation from a linear parameter-varying perspective, the method provides affine quadratic stability certificates and enables structured parameter-efficient fine-tuning. Furthermore, we design adapters based on Hypernets, Chebyshev polynomials, and DCT kernels to perform low-rank auxiliary mappings over frozen backbones. Experimental results validate the stability of timestep-conditioned memory profiles, with the Chebyshev variant achieving superior performance by reducing the average validation loss to 2.55.

Abstract

State Space Models (SSMs) offer an efficient alternative to Transformers for sequence modeling, yet conditioning pre-trained SSMs for iterative generation typically operates outside the recurrent operator, through input injection or activation modulation. While such mechanisms expose the model to conditioning information, they leave the underlying temporal dynamics fixed. We introduce MaRK (Markov-adapted Recurrent Kernels), a dynamic operator-conditioning framework that maps context vectors directly into bounded modulations of a frozen SSM's recurrence ($A$), read-in ($B$), read-out ($C$), skip ($D$), and discretization ($Δ$) parameters. Viewed through the lens of LPV-SSM systems, MaRK induces a context-indexed family of Markov parameter sequences, allowing each diffusion timestep to reshape the model's input-output memory kernel. We instantiate MaRK on a frozen 111M-parameter Hydra SSM backbone and study three adapter geometries: Hypernet, Chebyshev polynomial, and Discrete Cosine Transform kernels. Since these adapters modify the Markov parameter sequence through low-rank auxiliary maps on the frozen backbone, parameter-efficient fine-tuning arises as a structural consequence of the adaptation mechanism itself, requiring only 6.3--11M trainable auxiliary parameters to transition from a bidirectional objective to an iterative diffusion regime. The bounded recurrence parameterization further yields an analytic Affine Quadratic Stability certificate for the modulated recurrence. Through synthetic LPV recovery experiments and Markov-operator diagnostics, we show that MaRK recovers coordinate-invariant temporal operators under matched assumptions and produces distinct, stable timestep-conditioned memory profiles. Empirically, the Chebyshev variant yields the strongest performance, achieving an average validation loss of 2.55, followed by the DCT (2.59) and Hypernet (3.77) geometries.

Problem

• State Space Models
• Dynamic Operator Conditioning
• Iterative Generation
• Temporal Dynamics
• Diffusion Models

Innovation

• State Space Models
• Dynamic Operator Conditioning
• Parameter-Efficient Fine-Tuning
• Markov Parameter Sequences
• Stability Certificate