Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

2026-09-28

AI Summary

This study addresses the computational bottleneck of nonlinear sequence models, which lack the compositional structure necessary for efficient parallel computation akin to linear state space models (SSMs). We propose RiccatiSSM, which exploits the Möbius transformation properties of Riccati differential equations to design the first strictly composable state-dependent nonlinear dynamics. This formulation enables exact solutions via associative parallel scan algorithms without requiring iterative linearization. Coupled with constrained parameterization techniques, the proposed method achieves performance comparable to baseline models on long-sequence tasks while significantly reducing runtime by 22%–33% compared to LrcSSM, effectively overcoming the computational limitations of nonlinear SSMs.

Abstract

State space models (SSMs) achieve efficient sequence processing because their affine state updates are closed under composition and can therefore be evaluated with an associative parallel scan. Nonlinear recurrent models can provide richer, state-dependent dynamics, but generally lose this compositional structure: parallel evaluation then requires iterative methods that repeatedly linearize and scan the recurrence. We ask, what state-dependent nonlinear dynamics can be designed to remain exactly composable? We answer by introducing RiccatiSSM, a nonlinear SSM, in which each state dimension follows an input-conditioned Riccati differential equation. Its quadratic state dependence makes the local Jacobian explicitly state-dependent, while its exact per-step flow under piecewise-constant inputs is a M\"obius transformation. Since M\"obius maps are closed under composition and compose through $2\times 2$ matrix multiplication, the complete nonlinear state trajectory can be evaluated exactly with a single associative parallel scan, without iterative linearization. We further derive a constrained parameterization that ensures bounded, contractive dynamics, and avoids poles in the fractional-linear state update. Across long-sequence classification, regression, and forecasting tasks, RiccatiSSM achieves competitive predictive performance while reducing runtime by $22{-}33\%$ compared to the nonlinear LrcSSM under matched architectures. These results demonstrate that state-dependent nonlinear dynamics can retain exact composability and be evaluated efficiently within a single parallel scan.

Problem

• State Space Models
• Nonlinear Sequence Modeling
• Parallelization
• Composability
• Riccati Differential Equation

Innovation

• Riccati State Space Model
• Nonlinear Sequence Modeling
• Parallel Scan
• Möbius Transformation
• Exact Composability