design estimators

Adaptive filters lack a unified theoretical foundation. Method: This paper proposes a general Bayesian recursive inference framework, modeling observation noise as Gaussian or Laplacian to systematically derive classical algorithms—including LMS, NLMS, and Kalman filtering—as well as a novel family of robust filters. Contribution/Results: It establishes, for the first time, a unifying Bayesian interpretation encompassing both conventional and robust adaptive filters. Compared to conventional sign-error methods, the proposed algorithms exhibit superior robustness and convergence under Laplacian noise. The framework integrates state-space modeling, probabilistic noise characterization, and simplified structural analysis, ensuring both interpretability and extensibility. Numerical experiments demonstrate the algorithms’ enhanced performance in non-Gaussian noise environments. Overall, this work provides a rigorous, unified Bayesian theoretical basis for the design and analysis of adaptive filters.

This paper investigates the convergence of constant-step-size stochastic approximation (SA) algorithms under Markovian noise, focusing on root-finding—i.e., solving (f( heta^*) = 0)—and precisely characterizing the inherent bias and covariance error. To overcome the limitations of the classical i.i.d. noise assumption, we propose a joint parameter-perturbation process framework grounded in geometric ergodicity. This enables the first systematic analysis revealing a non-zero steady-state bias induced by “memory effects” in Markov noise, for which we derive a closed-form expression. Concurrently, we establish an explicit (O(alpha)) upper bound on the covariance error, quantifying how Markov dependence amplifies estimation error. Our theoretical results rigorously apply to temporal modeling settings such as TD-learning, and are corroborated by numerical experiments.

This work addresses the robustness of mean estimation in statistical learning under three concurrent challenges: adversarial data contamination, heavy-tailed distributions, and differential privacy constraints. Methodologically, it unifies robust statistics, high-dimensional geometry, stochastic optimization, and differential privacy theory to establish the first conceptual and algorithmic bridge across distinct robustness paradigms. Key technical abstractions—including iterative filtering, covariance trimming, and fractional gradient descent—are identified as common algorithmic primitives. The paper proposes a suite of computationally efficient estimators achieving statistically optimal convergence rates; each attains the information-theoretic lower bound under all three constraint classes simultaneously. By reconciling theoretical tightness with practical efficiency, this framework advances robust mean estimation from ad hoc heuristics toward a principled, unified design paradigm.

This work addresses discrete-time nonlinear optimal control problems by unifying classical algorithms—including gradient descent, Gauss–Newton, Newton’s method, and differential dynamic programming (DDP)—within a differentiable programming framework. Methodologically, it introduces the first modular, end-to-end differentiable algorithm template library built upon linear/quadratic approximations (e.g., LQR), enabled by automatic differentiation. Theoretically, it provides a unified derivation of computational complexity and sufficient optimality conditions across all methods. Practically, it incorporates adaptive line search and regularization strategies, and validates efficacy on benchmark tasks such as autonomous racing with a bicycle model. All implementations are open-sourced, demonstrating both efficient gradient propagation and strong generalization across diverse control problems.

This work addresses the challenge of nonlinear parameter estimation in Wiener-type state-space models by proposing a fixed-point iteration-based dual estimator framework. The approach couples two affine minimum mean square error (MMSE) estimators to separately handle unknown parameters and latent states, while introducing dynamic basis statistics (DBS) to efficiently summarize information from nonlinear basis functions. The resulting dual state-parameter and dual basis-parameter estimators alternately update their prior information, enabling stable and efficient nonlinear learning. Extensive Monte Carlo experiments demonstrate that the dual state-parameter estimator significantly outperforms existing methods—including purely affine estimators, particle Gibbs, and expectation-maximization (EM) variants of sequential Monte Carlo algorithms—in terms of parameter mean square error.

This work addresses the challenge that nonlinear Kalman filters—such as the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF)—often struggle to balance robustness and accuracy due to a lack of systematic design principles. To this end, the paper introduces a covariance compensation framework that quantifies the deviation from EKF’s covariance prediction and establishes design criteria for performance improvement. It presents, for the first time, the concept of covariance compensation along with three core guidelines: invariance under orthogonal transformations, sufficient compensation relative to the EKF baseline, and a preference for underconfident compensation magnitudes. Through theoretical analysis and numerical experiments, the study demonstrates that adherence to these principles significantly enhances estimation accuracy and reveals that commonly adopted fixed-parameter strategies in the literature are generally suboptimal.

This work addresses the challenge that real-world data in generalized linear models often violate the independent and identically distributed (i.i.d.) assumption. Under the relaxed assumption that the design matrix is orthogonally invariant—meaning its singular vectors are uniformly distributed while singular values remain arbitrary—the paper proposes an efficient parameter estimation method combining optimal spectral initialization with Approximate Message Passing (AMP). The proposed approach achieves the information-theoretically optimal sample complexity for weak recovery and attains the fundamental lower bound on estimation error, thereby extending beyond the classical i.i.d. Gaussian design setting. Rigorous theoretical analysis provides strong performance guarantees, and numerical experiments confirm both the algorithm’s effectiveness and the accuracy of the theoretical predictions on orthogonally invariant as well as more general correlated data.

This work addresses the challenge of heavy-tailed noise severely degrading accuracy and stability in distributed parameter estimation within densely deployed systems such as the Internet of Things. The authors propose a robust distributed estimation algorithm grounded in a nonlinear consensus+innovations framework, incorporating general nonlinear functions in both consensus and innovation updates to mitigate the adverse effects of heavy-tailed noise. For the first time under heavy-tailed noise conditions, the paper establishes theoretical guarantees of almost sure convergence and asymptotic normality for the proposed algorithm, revealing a quantitative trade-off between noise decay rates and network connectivity. The method achieves both estimation consistency and asymptotic optimality, thereby providing a novel theoretical foundation for highly robust distributed learning.

This work proposes a class of generalized Hessian estimators based on Random Direction Stochastic Approximation (RDSA) for zeroth-order optimization settings where only noisy function evaluations are available. By leveraging multi-point function measurements, the method constructs higher-order-accurate Newton-type update directions that significantly reduce estimation bias. Theoretically, the estimator is shown to be asymptotically unbiased, and the associated stochastic Newton algorithm is proven to converge, with both asymptotic and non-asymptotic analyses provided. These results extend the theoretical foundations of zeroth-order second-order optimization. Numerical experiments further demonstrate the effectiveness of the multi-point measurement strategy in enhancing optimization performance.