Weighted least squares estimation by multivariate-dependent weights for linear regression models

2026-01-15

AI Summary

This study addresses the challenge of heteroscedasticity in multivariate linear regression, where multiple explanatory variables jointly influence the error variance—a scenario inadequately handled by conventional univariate weighting approaches. The authors propose a novel multivariate-dependent weighted least squares estimator that constructs a linear combination of regressors to maximize its Spearman rank correlation with the absolute residuals, thereby capturing the joint effect of covariates on heteroscedasticity. This combined with a maximum likelihood framework to model the variance structure, the method uniquely integrates multivariate information into the weighting function. The resulting estimator is shown to be consistent and asymptotically normal. Extensive simulations and real-world applications—using Chinese household consumption and Boston housing data—demonstrate its superior performance over traditional methods in parameter estimation accuracy, prediction precision (measured by MAE and RSE), confidence interval coverage, and generalization, particularly under high volatility conditions.

Technology Category

• Machine Learning: Ensemble Methods
• Intelligent Robots: State Estimation
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Multivariate linear regression models often face the problem of heteroscedasticity caused by multiple explanatory variables. The weighted least squares estimation with univariate-dependent weights has limitations in constructing weight functions. Therefore, this paper proposes a multivariate dependent weighted least squares estimation method. By constructing a linear combination of explanatory variables and maximizing their Spearman rank correlation coefficient with the absolute residual value, combined with maximum likelihood method to depict heteroscedasticity, it can comprehensively reflect the trend of variance changes in the random error and improve the accuracy of the model. This paper demonstrates that the optimal linear combination exponent estimator for heteroscedastic volatility obtained by our algorithm possesses consistency and asymptotic normality. In the simulation experiment, three scenarios of heteroscedasticity were designed, and the comparison showed that the proposed method was superior to the univariate-dependent weighting method in parameter estimation and model prediction. In the real data applications, the proposed method was applied to two real-world datasets about consumer spending in China and housing prices in Boston. From the perspectives of MAE, RSE, cross-validation, and fitting performance, its accuracy and stability were verified in terms of model prediction, interval estimation, and generalization ability. Additionally, the proposed method demonstrated relative advantages in fitting data with large fluctuations. This study provides an effective new approach for dealing with heteroscedasticity in multivariate linear regression.

Problem

• heteroscedasticity
• multivariate linear regression
• weighted least squares
• explanatory variables
• variance changes

Innovation

• multivariate-dependent weights
• weighted least squares
• heteroscedasticity
• Spearman rank correlation
• linear regression