Fast and Efficient Implementation of the Maximum Likelihood Estimation for the Linear Regression with Gaussian Model Uncertainty

2025-07-15

AI Summary

This paper studies maximum likelihood estimation (MLE) for linear regression when the measurement matrix is a random variable—potentially rank-deficient in expectation—unifying overdetermined and underdetermined settings. To overcome the non-convexity and computational intractability of conventional MLE under rank deficiency, we propose an equivalent convex optimization framework and rigorously establish its convexity and strong duality—surpassing prior results limited to quasiconvexity. Theoretically, we reveal that randomness in the measurement matrix induces a novel statistical advantage in the underdetermined regime, improving parameter estimation accuracy. We further develop the generalized Random-Variable Maximum-Likelihood (RV-ML) algorithm, integrating Lagrangian duality analysis with efficient convex optimization techniques to enable unified, fast, and robust solutions. Extensive numerical experiments validate both the theoretical insights and the algorithm’s superiority over existing approaches.

Technology Category

• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

The linear regression model with a random variable (RV) measurement matrix, where the mean of the random measurement matrix has full column rank, has been extensively studied. In particular, the quasiconvexity of the maximum likelihood estimation (MLE) problem was established, and the corresponding Cramer-Rao bound (CRB) was derived, leading to the development of an efficient bisection-based algorithm known as RV-ML. In contrast, this work extends the analysis to both overdetermined and underdetermined cases, allowing the mean of the random measurement matrix to be rank-deficient. A remarkable contribution is the proof that the equivalent MLE problem is convex and satisfies strong duality, strengthening previous quasiconvexity results. Moreover, it is shown that in underdetermined scenarios, the randomness in the measurement matrix can be beneficial for estimation under certain conditions. In addition, a fast and unified implementation of the MLE solution, referred to as generalized RV-ML (GRV-ML), is proposed, which handles a more general case including both underdetermined and overdetermined systems. Extensive numerical simulations are provided to validate the theoretical findings.

Problem

• Extends MLE analysis to rank-deficient measurement matrices
• Proves convexity and strong duality in MLE problem
• Develops fast unified MLE solver for under/overdetermined systems

Innovation

• Convex MLE for rank-deficient matrix
• Generalized RV-ML unified algorithm
• Beneficial randomness in underdetermined cases