filter tuning

This work addresses the challenge that nonlinear Kalman filters—such as the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF)—often struggle to balance robustness and accuracy due to a lack of systematic design principles. To this end, the paper introduces a covariance compensation framework that quantifies the deviation from EKF’s covariance prediction and establishes design criteria for performance improvement. It presents, for the first time, the concept of covariance compensation along with three core guidelines: invariance under orthogonal transformations, sufficient compensation relative to the EKF baseline, and a preference for underconfident compensation magnitudes. Through theoretical analysis and numerical experiments, the study demonstrates that adherence to these principles significantly enhances estimation accuracy and reveals that commonly adopted fixed-parameter strategies in the literature are generally suboptimal.

Conventional Kalman filtering and minimum-variance unbiased (MVU) estimation fail in structural health monitoring when sensor configurations are non-ideal—e.g., rank-deficient feedthrough matrices, direct transmission paths, or unmodeled noise/parameter uncertainties. Method: This paper proposes a general adaptive filtering and smoothing framework integrating MVU principles, state-augmented filtering, and online self-calibration—without requiring fictitious input models or full-rank feedthrough assumptions. Contribution/Results: Validated for the first time in physical experiments using a five-story shear-frame shake-table setup under multiple impact excitations, the method demonstrates robustness and real-time adaptability under realistic sensor noise and structural parameter uncertainty. It achieves high-accuracy joint estimation of system states and unknown inputs, significantly extending the engineering applicability boundary of general unknown-input estimation methods.

Traditional Kalman filters for nonlinear systems—such as the Extended, Unscented, and Cubature Kalman Filters—suffer from overconfident state estimates, underestimated covariances, and degraded accuracy due to nonlinear measurement functions. This paper identifies, for the first time, the intrinsic bias in covariance propagation underlying these deficiencies and proposes a general, plug-and-play correction framework. Grounded in Bayesian estimation reformulation and explicit covariance propagation calibration, the framework is compatible with mainstream nonlinear Kalman filter variants. Theoretical analysis and extensive experiments across five canonical tasks demonstrate that the method reduces state estimation error by one to three orders of magnitude—particularly under low measurement noise—while significantly improving covariance fidelity and effectively mitigating overconfidence.

This work addresses the challenge of accurately estimating the filtering distribution (i.e., the state posterior) in high-dimensional nonlinear dynamical systems. To overcome the bias inherent in traditional ensemble Kalman filters (EnKF) under strong nonlinearity and their reliance on labor-intensive manual tuning, we propose an end-to-end learning framework grounded in variational inference. Specifically, the filter’s analysis step is modeled as a learnable, parameterized analysis mapping; key components—including gain computation, covariance inflation, and localization—are jointly optimized via a variational objective. This constitutes the first systematic integration of variational inference into filter design, enabling unified modeling and automatic calibration of the analysis process. Experiments across diverse linear and nonlinear systems demonstrate that our method significantly reduces filtering bias, improves posterior estimation accuracy, and drastically diminishes dependence on manual parameter tuning.

To address the inefficiency and poor generalizability of manual hyperparameter tuning—particularly for learning rates—this paper proposes a dynamic online meta-optimization framework that formulates learning rate adaptation as a discounted cumulative regret minimization problem over time. The method employs a gradient-based meta-update mechanism, enabling plug-and-play integration with any first-order optimizer (e.g., SGD, Adam) to achieve decoupled, real-time, adaptive step-size optimization. Key contributions include: (i) the first formalization of meta-optimization as discounted regret minimization; and (ii) a low-complexity variant that preserves theoretical rigor while ensuring computational efficiency and strong generalization. Experiments across diverse tasks demonstrate faster convergence, enhanced robustness to initialization and task heterogeneity, competitive performance against hand-tuned optimal schedulers, and significantly lower computational overhead compared to conventional hyperparameter search methods.

This work investigates the signal tracking performance of the ensemble Kalman filter (EnKF) under partial observations, focusing on optimal state subspace dimension selection when the observation operator exhibits fixed, stochastic, or adaptive variability. We propose an error-feedback-driven adaptive sequential learning mechanism that dynamically determines, online, the minimal subspace dimension ensuring bounded filtering error—thereby achieving an optimal trade-off between observational complexity and estimation accuracy. A rigorous theoretical upper bound on the tracking error is derived as a function of observation stochasticity. Experiments demonstrate that the mechanism accurately identifies critical subspaces and maintains high-precision estimation while significantly reducing observational burden. This study establishes a systematic theoretical framework and provides a practical algorithm for subspace-adaptive EnKF design under incomplete observations.

This study addresses the limitations of conventional IMU-GPS fusion methods, which often struggle with nonlinear dynamics, stability, or computational efficiency, thereby failing to meet the demands of high-precision vehicle localization. To overcome these challenges, this work proposes an adaptive parameter-tuning framework based on particle swarm optimization (PSO), which, for the first time, enables joint optimization of multiple parameters (α, β, κ, Q, R) in the unscented Kalman filter (UKF). Evaluated across diverse driving scenarios in the CARLA simulation platform using a Tesla Model 3 vehicle model, the proposed approach achieves an 82.14% improvement in localization accuracy over manual tuning, reduces maximum IMU drift by 21,606.59 meters, and maintains a per-update computation time under 10 milliseconds. These results demonstrate a favorable balance of accuracy, robustness, and real-time performance, highlighting its practical applicability.

This work proposes a novel experimental design framework for dynamic systems that addresses two key limitations of existing approaches: the neglect of process noise and the reliance on unknown true parameters for computing the Fisher information matrix (FIM). By integrating Bayesian averaging with an adaptive updating mechanism, the method jointly accounts for both process and measurement noise through Kalman filtering. The FIM is computed via Bayesian averaging over the parameter prior and is continuously updated in real time as new data become available, thereby optimizing subsequent experimental inputs. This approach achieves, for the first time, robust and real-time experimental design in linear dynamic systems with process noise without requiring knowledge of the true system parameters, significantly enhancing both the information efficiency and robustness of system identification.

Uncertainty propagation in neural network surrogate models of nonlinear dynamical systems remains challenging, particularly due to the computational burden and approximation errors inherent in Monte Carlo methods. Method: This paper proposes a novel state estimation algorithm integrating analytical moment computation with assumed density filtering (ADF) and Rauch–Tung–Striebel (RTS) smoothing. It leverages newly derived closed-form expressions for the mean and covariance of deep neural network outputs under Gaussian input distributions, eliminating reliance on Monte Carlo sampling. Cross-entropy is adopted as a distribution-aware metric for evaluating filter/smoothing performance, offering greater sensitivity to probabilistic fidelity than conventional RMSE. Contribution/Results: The method achieves significantly improved state estimation accuracy on stochastic Lorenz and Wiener systems. Moreover, enhanced uncertainty quantification directly translates into superior performance of downstream linear-quadratic regulator (LQR) control, demonstrating the practical benefits of analytically propagated uncertainties in closed-loop applications.

This work addresses the performance degradation and miscalibrated uncertainty quantification of traditional Kalman filters under misspecified observation noise, particularly in the presence of heavy-tailed outliers. It introduces diffusion score matching into the ensemble Kalman filter framework for the first time, refining the analysis step to achieve robustness against non-Gaussian observation noise while preserving reliable uncertainty quantification. The approach is developed for linear Gaussian systems, yielding conjugate closed-form updates, and is extended through generalized Bayesian inference, stochastic–deterministic coupling, and localization to produce robust variants of EnKF, ESRF, and LETKF. Experiments on target tracking and Lorenz-63/96 systems demonstrate superior data assimilation performance under nonlinear dynamics and heavy-tailed noise, supported by theoretical guarantees of high-dimensional consistency.