On the inverse of covariance matrices for unbalanced crossed designs

2025-12-09

AI Summary

In unbalanced crossed designs, the response covariance matrix $mathbf{V}$ in linear mixed models lacks a closed-form inverse $mathbf{V}^{-1}$, hindering likelihood-based estimation and inference. To address this, we derive the first exact low-rank correction formula for $mathbf{V}^{-1}$. We propose a spectral decomposition construction method based on the Khatri–Rao product, establishing a unified approximation framework that bridges asymptotically balanced to arbitrarily unbalanced settings. The method ensures high accuracy, numerical stability, and computational efficiency, substantially reducing the complexity of likelihood evaluation. Simulation studies demonstrate its robust accuracy and scalability across diverse degrees of imbalance. This work provides both theoretical foundations and practical tools for statistical modeling and inference in unbalanced crossed experiments.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

This paper addresses a long-standing open problem in the analysis of linear mixed models with crossed random effects under unbalanced designs: how to find an analytic expression for the inverse of $mathbf{V}$, the covariance matrix of the observed response. The inverse matrix $mathbf{V}^{-1}$ is required for likelihood-based estimation and inference. However, for unbalanced crossed designs, $mathbf{V}$ is dense and the lack of a closed-form representation for $mathbf{V}^{-1}$, until now, has made using likelihood-based methods computationally challenging and difficult to analyse mathematically. We use the Khatri--Rao product to represent $mathbf{V}$ and then to construct a modified covariance matrix whose inverse admits an exact spectral decomposition. Building on this construction, we obtain an elegant and simple approximation to $mathbf{V}^{-1}$ for asymptotic unbalanced designs. For non-asymptotic settings, we derive an accurate and interpretable approximation under mildly unbalanced data and establish an exact inverse representation as a low-rank correction to this approximation, applicable to arbitrary degrees of unbalance. Simulation studies demonstrate the accuracy, stability, and computational tractability of the proposed framework.

Problem

• Finding analytic inverse for covariance matrices in unbalanced crossed designs
• Enabling likelihood-based estimation for crossed random effects models
• Providing computationally tractable approximations for arbitrary unbalance degrees

Innovation

• Using Khatri-Rao product to represent covariance matrix
• Deriving exact spectral decomposition via modified covariance matrix
• Providing low-rank correction for inverse approximation