lmmse derivation

Channel estimation in one-bit quantized massive MIMO systems suffers from severe accuracy degradation due to coarse quantization. Method: Breaking the limitations of conventional Bussgang linear approximation, this work establishes the first rigorous minimum mean-square error (MMSE) channel estimation framework grounded in orthant probabilities of multivariate normal distributions. It integrates Bussgang decomposition, statistical channel modeling, and asymptotic analysis to enable efficient MMSE computation. Contribution/Results: Theoretically, we derive necessary and sufficient conditions for the optimality of the Bussgang-linear MMSE (BLMMSE) estimator and obtain a closed-form, analytically tractable MMSE estimator in the high-dimensional regime—revealing the fundamental gap between linear approximation and global optimality. Experiments under typical massive MIMO configurations demonstrate substantial estimation error reduction. Moreover, we explicitly characterize the applicability boundary of BLMMSE and quantify its performance loss as a function of channel correlation structure and pilot design.

Existing pedagogical treatments of linear regression often lack rigorous theoretical unification across geometric, algebraic, and statistical perspectives, and seldom establish formal optimality guarantees or bridge frequentist and Bayesian interpretations. Method: This paper constructs a rigorous theoretical framework for linear regression targeting readers familiar with ordinary least squares (OLS), integrating geometric projection, matrix algebra, and statistical inference. It introduces Gaussian noise assumptions to derive maximum likelihood estimation, exact sampling distributions (t- and F-tests, confidence intervals), and Bayesian linear regression with closed-form posterior inference. Contribution/Results: We provide the first rigorous proof that OLS achieves the Cramér–Rao lower bound among unbiased linear estimators—establishing its theoretical optimality. The work unifies frequentist and Bayesian paradigms, yielding analytically tractable inference tools. The resulting verifiable statistical pipeline serves as an interpretable linear foundation and error-analysis benchmark for nonlinear models, including deep learning.

To address the sequential determination of model order and insufficient robustness in high-dimensional line spectral estimation (LSE), this paper proposes the Bilinear Generalized LSE (BiG-LSE) method. BiG-LSE iteratively approximates the nonlinear observation model via Taylor expansion into a bilinear form and employs Expectation Propagation (EP) for joint Bayesian inference of frequencies, model order, and noise variance. It is the first LSE framework enabling fully automatic order selection and rigorous uncertainty quantification. To enhance sequential estimation accuracy, it adopts the von Mises distribution to model frequency priors. Furthermore, low-rank compression of posterior log-density messages significantly reduces computational complexity. Simulation and real-world experiments demonstrate that BiG-LSE achieves estimation accuracy comparable to state-of-the-art methods while exhibiting superior robustness and practicality under high-dimensional nonlinear measurements.

This paper addresses the general parametric estimation problem without distributional assumptions, aiming to construct estimators robust to both model misspecification and complex data dependence structures. We propose a minimum distance estimation framework based on the Maximum Mean Discrepancy (MMD), establishing— for the first time—its statistical consistency under non-i.i.d. sampling and model misspecification, and providing theoretical convergence guarantees for stochastic gradient descent optimization. Theoretically, the estimator exhibits intrinsic robustness to temporal dependence, outliers, and distributional shifts. Numerical experiments demonstrate its superior performance over classical M-estimators under data contamination and intricate time-series settings. Our core contribution lies in unifying the characterization of the MMD estimator’s generalization error and robustness limits, thereby offering a novel paradigm for reliable inference with nonstandard data.

This paper addresses the computational burden of hyperparameter optimization in regularized estimation for system identification. To this end, we propose two novel parameter-free estimators: a generalized Bayesian estimator and a closed-form biased estimator. Theoretically, we construct, for the first time, a family of hyperparameter-free estimators whose excess mean squared error (MSE) strictly matches that of the optimal empirical Bayes regularized estimator. Methodologically, leveraging large-sample asymptotics, ridge regression modeling, and bias–variance trade-off analysis, we derive analytical closed-form solutions—bypassing iterative optimization entirely. Numerical experiments demonstrate that the proposed estimators achieve MSE performance comparable to the best empirical Bayes method, while reducing computational time significantly. This makes them particularly suitable for real-time applications and resource-constrained environments.

This work addresses the instability of maximum a posteriori (MAP) estimation in two-dimensional blind deconvolution—a canonical blind inverse problem—where non-convexity of the objective function and non-identifiability of solutions often lead to unreliable performance. Under fully controlled experimental conditions, the study systematically compares MAP algorithms employing exact priors against a structurally near-optimal Tikhonov-regularized linear minimum mean square error (LMMSE) estimator. The results demonstrate that LMMSE not only significantly outperforms MAP methods, which require meticulous hyperparameter tuning, but also serves as a robust initialization strategy for MAP optimization, effectively reducing its sensitivity to regularization parameters and enhancing both stability and reconstruction quality. These findings establish LMMSE as playing a dual role in blind deconvolution: a high-performance baseline and a reliable starting point for iterative refinement.

The selection of the regularization parameter in low-rank minimum mean square error (MMSE) filters typically relies on empirical tuning or cross-validation, lacking rigorous theoretical guidance. Method: This paper establishes, for the first time, an analytical mapping between the regularization parameter and the filter rank, and proposes a Kronecker-structured automatic parameter selection method. Within the Bayesian MMSE framework, it analytically models the regularization term to enable adaptive, closed-form parameter determination—eliminating the need for grid search or retraining. Contribution/Results: Theoretical analysis uncovers the intrinsic coupling between regularization strength and effective rank. Simulations demonstrate that the proposed method significantly improves estimation accuracy across varying signal-to-noise ratios (SNRs), achieving average SNR gains of 1.8–3.2 dB over conventional Tikhonov regularization and cross-validation approaches, while maintaining high computational efficiency and robustness.

This study addresses the minimax estimation problem under a bounded normal mean model. The authors propose a novel approach based on the stochastic mirror ascent algorithm to approximate the least favorable prior distribution by solving a concave maximization problem, and then adopt its Bayes estimator as an approximate minimax estimator. This work is the first to introduce stochastic mirror ascent into this class of estimation problems and provides theoretical guarantees for the approximation accuracy. Numerical experiments demonstrate that the proposed estimator reduces risk by 6% to nearly 18% compared to the classical minimax linear estimator. Furthermore, the method is successfully applied to impulse response coefficient estimation, confirming its practical effectiveness.

This work addresses the implicit regularization structure and convergence properties of plug-and-play (PnP) methods employing minimum mean squared error (MMSE) denoisers. We establish, for the first time, that MMSE-PnP is equivalent to explicit optimization with a 1-weakly convex regularizer given by the upper Moreau envelope of the negative log marginal density. Building on this insight, we derive the first non-asymptotic sublinear convergence rate for PnP gradient descent—removing restrictive asymptotic or strong convexity assumptions required in prior analyses. Our approach integrates weak convexity theory, proximal operator analysis, and implicit regularization modeling. We validate the theoretical convergence behavior and practical efficacy through synthetic 1D experiments and real-world inverse problems, including image deblurring and CT reconstruction. Key contributions include: (i) characterizing the precise implicit regularizer induced by MMSE denoisers; (ii) providing the first non-asymptotic convergence guarantee for PnP gradient descent; and (iii) demonstrating tight alignment between theory and empirical performance.

This work addresses the marked degradation in robustness of Maronna’s and Tyler’s M-estimators under high-dimensional observations in the presence of clustered outliers. To mitigate this issue, we propose a novel M-estimation approach that enhances the stability of scatter matrix estimation by shrinking the precision matrix toward the identity matrix. We establish sufficient conditions for the existence of the proposed estimator, derive upper and lower bounds on its breakdown point, and provide both a statistical interpretation and an efficient numerical optimization algorithm for its implementation. Experimental results demonstrate that the proposed method substantially improves robustness and estimation accuracy on high-dimensional data contaminated with clustered outliers.