A new hierarchical distribution on arbitrary sparse precision matrices

2025-06-11

AI Summary

This paper addresses Bayesian estimation of sparse precision matrices. To jointly enforce sparsity and positive definiteness, we propose a novel hierarchical prior defined on the space of sparse symmetric positive-definite matrices, based on a structured Cholesky decomposition (S-Bartlett decomposition), thereby circumventing the intractable normalizing constant inherent in Wishart-type priors. Sparsity in the underlying graph structure is explicitly modeled via a spike-and-slab prior. We further develop a Dual Averaging Hamiltonian Monte Carlo algorithm for efficient, adaptive posterior sampling. The framework naturally integrates with generalized linear models, enabling graph-structured learning under non-Gaussian response distributions. Empirical evaluations on both synthetic and real-world datasets demonstrate that our method consistently outperforms the G-Wishart prior in terms of accuracy, robustness, and flexibility—particularly in precision matrix estimation and graph structure recovery.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Graphical Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We introduce a general strategy for defining distributions over the space of sparse symmetric positive definite matrices. Our method utilizes the Cholesky factorization of the precision matrix, imposing sparsity through constraints on its elements while preserving their independence and avoiding the numerical evaluation of normalization constants. In particular, we develop the S-Bartlett as a modified Bartlett decomposition, recovering the standard Wishart as a particular case. By incorporating a Spike-and-Slab prior to model graph sparsity, our approach facilitates Bayesian estimation through a tailored MCMC routine based on a Dual Averaging Hamiltonian Monte Carlo update. This framework extends naturally to the Generalized Linear Model setting, enabling applications to non-Gaussian outcomes via latent Gaussian variables. We test and compare the proposed S-Bartelett prior with the G-Wishart both on simulated and real data. Results highlight that the S-Bartlett prior offers a flexible alternative for estimating sparse precision matrices, with potential applications across diverse fields.

Problem

• Defining distributions over sparse symmetric positive definite matrices
• Developing S-Bartlett prior for sparse precision matrix estimation
• Extending framework to non-Gaussian outcomes via latent variables

Innovation

• Cholesky factorization for sparse matrices
• Spike-and-Slab prior for graph sparsity
• Dual Averaging Hamiltonian Monte Carlo