frequency-aware state space modeling

In low-data regimes, state space models (SSMs) suffer from poor sample efficiency and weak generalization due to rigid, task-agnostic inductive biases. To address this, this work formally characterizes the inductive bias of linear time-invariant SSMs as an “SSM-induced kernel” and proposes a task-dependent initialization method based on spectral matching: prior to training, model parameters are dynamically aligned with the spectral structure of the target task via frequency-domain analysis and kernel theory. This approach incurs no additional training overhead while significantly improving few-shot generalization. Empirical evaluation across multiple real-world benchmarks demonstrates substantial performance gains over strong baselines—particularly in ultra-low-data regimes—establishing a new paradigm for efficient and scalable SSM design.

Addressing the challenge of balancing long-term prediction stability and computational efficiency in modeling long sequences, this paper proposes the Linear Oscillatory State Space (LinOSS) model. LinOSS is grounded in the forced harmonic oscillator differential equation and employs a nonnegative diagonal state matrix to ensure lightweight, intrinsic stability. It leverages implicit-explicit discretization and a fast associative parallel scan algorithm to enable efficient, scalable sequence processing. Theoretically, LinOSS is proven to possess universal function approximation capability and preserve time-reversal symmetry. In long-range forecasting tasks up to 50k steps, LinOSS achieves nearly 2× higher accuracy than Mamba and 2.5× higher than LRU, significantly outperforming existing state-of-the-art methods.

This work addresses the limited interpretability of existing state space models regarding their long-range dependency mechanisms, particularly the unclear relationship between modeling capacity and architectural design in real-world tasks. Focusing on the S4D model, we present the first systematic analysis of its kernel behavior in the context of source code vulnerability detection. By integrating time-domain and frequency-domain analyses, we demonstrate that S4D can function as a low-pass, band-pass, or high-pass filter depending on its architectural configuration. This finding reveals that the model’s ability to capture long-range dependencies is profoundly influenced by its architecture, thereby offering both theoretical insights and concrete guidance for designing more effective state space models.

This work addresses the fundamental question of how state space models (SSMs) can perform zero-shot in-context learning to predict the next state of arbitrary dynamical systems without parameter fine-tuning. Method: We propose HiPPO-Prophecy, a novel weight construction method for SSMs grounded in the HiPPO framework, enabling both continuous- and discrete-time modeling. Our approach theoretically establishes that continuous SSMs can asymptotically approximate the derivative of any input signal, and we derive provable next-state prediction guarantees for discrete SSMs. Through rigorous signal-theoretic analysis and asymptotic error bound derivation, we obtain an explicit upper bound on the derivative approximation error. Results: Experiments demonstrate high-accuracy zero-shot state prediction across diverse dynamical systems. This work provides the first theoretical characterization of SSMs’ capacity for modeling dynamical system evolution in zero-shot settings, significantly enhancing both their theoretical interpretability and practical applicability.

Existing dynamical system forecasting models suffer from significant limitations in long-horizon prediction accuracy, modeling of long-range dependencies, capture of chaotic evolution, and extrapolation under scarce-data conditions. This paper introduces the first Mamba-enhanced state-space model (SSM) for physics-informed machine learning (PIML), innovatively embedding Mamba’s structured state evolution mechanism into a neural operator framework while integrating parameter remapping and quantitative systems pharmacology priors. We design a rigorous extrapolation benchmark encompassing chaotic systems and multiscale dynamics to systematically address generalization bottlenecks under long-range dependencies, strong nonlinearity, and data scarcity. Experiments demonstrate state-of-the-art performance across diverse interpolation and extrapolation tasks with the lowest computational overhead. In real-world evaluation of anticancer drug efficacy, the method achieves highly robust predictions using only minimal clinical data—marking a critical advance in interpretable, sample-efficient PIML for complex biological dynamics.

This work addresses the challenge that traditional time-invariant models struggle to effectively capture switching dynamics in time-varying systems. To overcome this limitation, the authors propose a neural network–based time-varying state-space model that incorporates a learnable dictionary of time-varying basis functions. This design flexibly represents diverse temporal evolution patterns of system dynamics while maintaining manageable computational complexity, substantially enhancing the model’s capacity to capture switching sequences. The study further reveals an optimal allocation strategy for time-varying degrees of freedom across model components. Experimental results demonstrate that the proposed model consistently outperforms existing time-invariant approaches on both synthetic switching systems and speech denoising tasks, confirming its effectiveness and strong generalization capability.

研究了混合专家状态空间模型在处理复杂动态系统时的表现，发现增加专家数量反而导致性能下降，通过相空间分析揭示了误差降低可能掩盖了对真实动态的不准确建模。

This work investigates the theoretical underpinnings of memorization and overfitting in stochastic interpolation generative models. Focusing on continuous-time stochastic differential equations and their Euler discretization, it provides the first rigorous theoretical definitions of overfitting and underfitting in generative modeling and derives closed-form expressions for the optimal velocity field and score function. The analysis reveals that generated samples can be expressed as training samples perturbed by three controllable error terms, whose bias is jointly determined by the discretization step size and estimation error. Synthetic experiments corroborate the theoretical prediction that generated samples cluster around the training data distribution, highlighting the critical roles of error accumulation and noise modeling in the model’s reconstruction capability.

This work addresses the challenge of simultaneously achieving stability, interpretability, and generalization in time series forecasting by proposing a novel architecture that integrates learnable Koopman operators with Transformer-based backbones such as PatchTST, Informer, and Autoformer. By designing four variants of the Koopman operator, the method enables explicit control over the spectral properties, stability, and rank of the linear transition operator within deep forecasting models for the first time, allowing flexible interpolation between strictly stable and unconstrained dynamics. Experiments demonstrate that the approach significantly improves the bias-variance trade-off, numerical conditioning, and interpretability of latent dynamics across multi-horizon forecasting tasks, effectively combining theoretical guarantees with data-driven flexibility.

This work addresses the lack of a precise mathematical characterization of end-to-end input–output mappings in existing state space models (SSMs), such as S4D. By establishing a rigorous correspondence between S4D and analytically solvable networks of nonlinear oscillators, the authors embed S4D into a ring topology and derive, for the first time, an explicit operator expression for forward propagation. This derivation yields the first complete end-to-end analytical formulation for modern SSMs, revealing the underlying mechanism of information wave propagation through the ring structure and the interactions induced by nonlinear decoders. The resulting framework provides a clear physical interpretation and enhanced interpretability, and it generalizes across several mainstream SSM architectures.