Technology Category
Machine Learning: Probabilistic Circuits and Graphical ModelsReasoning under Uncertainty: Relational Probabilistic ModelsCognitive Modeling & Cognitive Systems: Neural Spike Coding
Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problems
🤖 AI Summary
Modeling sparse priors under generic bases remains challenging for linear inverse problems. Method: This paper proposes a probabilistic sparse prior based on a Degenerate Gaussian Mixture Model (DGMM), integrated into a neural network architecture that ensures both Bayesian interpretability and end-to-end differentiability. For the first time, DGMM is embedded explicitly to realize Maximum A Posteriori (MAP) estimation, enabling joint supervised and unsupervised training. Results: Evaluated on multiple 1D inverse problems, the method consistently outperforms LASSO, group LASSO, iterative hard thresholding, and sparse coding in reconstruction mean squared error, while demonstrating strong robustness to noise and model mismatch. Key contributions include: (i) the first DGMM-based sparse modeling framework for generic bases; and (ii) the first differentiable neural integration of DGMM grounded in Bayesian semantics.
Technology Category
Application Category
📝 Abstract
In inverse problems it is widely recognized that the incorporation of a sparsity prior yields a regularization effect on the solution. This approach is grounded on the a priori assumption that the unknown can be appropriately represented in a basis with a limited number of significant components while most coefficients are close to zero. This occurrence is frequently observed in real-world scenarios, such as with piecewise smooth signals. In this study we propose a probabilistic sparsity prior formulated as a mixture of degenerate Gaussians, capable of modelling sparsity with respect to a generic basis. Under this premise we design a neural network that can be interpreted as the Bayes estimator for linear inverse problems. Additionally, we put forth both a supervised and an unsupervised training strategy to estimate the parameters of this network. To evaluate the effectiveness of our approach we conduct a numerical comparison with commonly employed sparsity-promoting regularization techniques, namely Least Absolute Shrinkage and Selection Operator (LASSO), group LASSO, iterative hard thresholding and sparse coding/dictionary learning. Notably, our reconstructions consistently exhibit lower mean square error values across all one-dimensional datasets utilized for the comparisons, even in cases where the datasets significantly deviate from a Gaussian mixture model.
Problem
Research questions and friction points this paper is trying to address.
Modeling sparsity with Gaussian mixture for inverse problems
Designing neural network as Bayes estimator for linear inverse problems
Comparing effectiveness with LASSO and other sparsity techniques
Innovation
Methods, ideas, or system contributions that make the work stand out.
Mixture of degenerate Gaussians for sparsity prior
Neural network as Bayes estimator
Supervised and unsupervised training strategies
🔎 Similar Papers
No similar papers found.
