kalman filtering

This work investigates the out-of-distribution (OOD) generalization of Recursive KalmanNet—specifically, its ability to robustly estimate both states and error covariance when test-time measurement dynamics significantly deviate from the training distribution. To address this, we propose a differentiable recurrent architecture that explicitly embeds the structural prior of Kalman filtering into a neural network, enabling joint end-to-end learning of system dynamics and noise statistics without requiring prior knowledge of noise distributions. By unifying sequential modeling with Bayesian filtering principles, the framework achieves data-driven, robust state estimation. Experiments demonstrate that our method maintains accurate state estimates and well-calibrated covariance predictions under unseen dynamics, substantially outperforming both classical Kalman filters (which assume known models and noise) and purely data-driven RNNs. It exhibits superior extrapolation capability and more reliable uncertainty quantification in OOD settings.

State estimation for stochastic dynamic systems under non-Gaussian noise remains challenging due to the breakdown of Gaussian assumptions underlying classical filters. Method: This paper proposes a Kalman Filter-guided Recurrent Neural Network (KF-RNN), the first to embed the Joseph form of the covariance update within an RNN architecture and jointly optimize a Gaussian negative log-likelihood objective, ensuring mathematical consistency in uncertainty propagation and statistical reliability. Contribution/Results: The method preserves recursive, real-time inference while delivering high-accuracy state estimates and rigorously calibrated error covariance quantification. Experiments under non-Gaussian white noise demonstrate a 32% reduction in state estimation error and a 47% decrease in covariance calibration error compared to baselines—outperforming both conventional Kalman filters and state-of-the-art deep estimators.

This work addresses variational state estimation by establishing, for the first time, a systematic theoretical framework grounded in dynamic programming. Methodologically, it introduces recursive forward and backward value functionals, yielding a variational dual-filter formulation analogous to classical Bayesian filtering and smoothing; it further reveals that these value functionals upper-bound the logarithm of the unnormalized posterior density—providing rigorous theoretical justification for variational approximation. A linear-complexity suboptimal variational filtering algorithm is then developed, balancing computational efficiency with estimation accuracy. The approach integrates dynamic programming, variational inference, and unnormalized density approximation, and achieves tractable recursive inference in jump Markov linear Gaussian systems via factorized Markov approximations. Simulation results demonstrate high-fidelity posterior approximation, favorable computational tractability, and superior estimation quality.

Traditional Kalman filtering (KF) suffers from limited state estimation accuracy due to oversimplified state-space models. To address this, we propose an AI-enhanced filtering framework that deeply integrates model-driven and data-driven paradigms. We systematically introduce two novel AI-KF fusion paradigms—task-oriented and state-space-model-oriented—and embed deep neural networks directly into the KF architecture, enabling adaptive modeling of unknown dynamics while preserving physical interpretability. Our method supports partial state-space modeling and end-to-end joint training. Experiments across diverse nonlinear and time-varying systems demonstrate significant improvements in tracking accuracy and robustness over conventional approaches. Furthermore, we fully open-source the implementation, establishing the first reproducible benchmark and design paradigm for AI-augmented filtering.

This work addresses the degraded accuracy and limited scalability of traditional ensemble Kalman filters (EnKF) in discrete-time nonlinear filtering under model misspecification or poor initialization. To overcome these limitations, the paper proposes the Exact Ensemble Kalman Filter (ExEnKF), which, for the first time within the EnKF framework, replaces Dirac measures with Gaussian approximations to more effectively explore the state space while preserving computational efficiency. Theoretical analysis establishes that ExEnKF converges to the optimal filter at a rate of $1/\sqrt{N}$, where $N$ is the ensemble size. Numerical experiments demonstrate its superior performance over standard EnKF and sequential Monte Carlo methods in highly stochastic, model-mismatched, and multiscale Lorenz-96 systems, particularly in robustly tracking unobservable hidden state components. This approach thus offers a practical, scalable, and accurate solution for high-dimensional nonlinear filtering.

This work addresses the challenge that nonlinear Kalman filters—such as the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF)—often struggle to balance robustness and accuracy due to a lack of systematic design principles. To this end, the paper introduces a covariance compensation framework that quantifies the deviation from EKF’s covariance prediction and establishes design criteria for performance improvement. It presents, for the first time, the concept of covariance compensation along with three core guidelines: invariance under orthogonal transformations, sufficient compensation relative to the EKF baseline, and a preference for underconfident compensation magnitudes. Through theoretical analysis and numerical experiments, the study demonstrates that adherence to these principles significantly enhances estimation accuracy and reveals that commonly adopted fixed-parameter strategies in the literature are generally suboptimal.

This study addresses online learning of the Kalman filter for output and state estimation in partially observable linear dynamical systems with unknown system models. The authors propose a unified algorithmic framework based on online optimization, incorporating a stochastic querying mechanism to handle limited observability. Their theoretical analysis establishes, for the first time, that sublinear regret in state estimation is unattainable without queries, yet a √T regret bound becomes achievable with a finite number of stochastic queries, revealing a fundamental trade-off between query complexity and regret. The proposed algorithm attains a logarithmic regret bound (log T) for output estimation and a √T regret bound for state estimation. Numerical experiments corroborate the theoretical findings and demonstrate the algorithm’s empirical effectiveness.

Traditional Kalman filtering is constrained by linear Gaussian assumptions, leading to suboptimal performance in nonlinear sensing scenarios such as Doppler radar and LiDAR, where mere parameter tuning cannot overcome inherent structural limitations. This work proposes the Kalman Evolve framework, which for the first time introduces algorithmic structure discovery into state estimation by jointly optimizing noise parameters and update structures. Leveraging large language models as structured priors over program space, the method employs program synthesis to generate interpretable, non-affine filtering algorithms that retain recursive form while adapting effectively to nonlinear dynamics. Experiments across diverse real-world and synthetic tracking tasks demonstrate up to a 12% reduction in root mean square error (RMSE), significantly outperforming strong existing baselines.

This work proposes a novel experimental design framework for dynamic systems that addresses two key limitations of existing approaches: the neglect of process noise and the reliance on unknown true parameters for computing the Fisher information matrix (FIM). By integrating Bayesian averaging with an adaptive updating mechanism, the method jointly accounts for both process and measurement noise through Kalman filtering. The FIM is computed via Bayesian averaging over the parameter prior and is continuously updated in real time as new data become available, thereby optimizing subsequent experimental inputs. This approach achieves, for the first time, robust and real-time experimental design in linear dynamic systems with process noise without requiring knowledge of the true system parameters, significantly enhancing both the information efficiency and robustness of system identification.