Technology Category
Machine Learning: Calibration & Uncertainty QuantificationCognitive Modeling & Cognitive Systems: Neural Spike CodingReasoning under Uncertainty: Graphical Models
Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks
🤖 AI Summary
This paper addresses persistent challenges in Bayesian neural networks (BNNs) for modeling natural phenomena: difficulty in characterizing structural uncertainty, over-parameterization, and limited interpretability. To this end, we propose input-skip latent binary Bayesian neural networks (input-skip LBBNNs). Our method introduces an “active path” mechanism that enables inputs to bypass intermediate layers and connect directly to any subsequent layer—or be automatically pruned—thereby encoding input-level structural priors and achieving extreme sparsity (>99.9% weight pruning). Crucially, it intrinsically yields theoretically grounded global and local explanations without post-hoc processing. On MNIST, the model attains 97% accuracy using only 935 parameters while demonstrating excellent uncertainty calibration. Its explanation fidelity and compression ratio surpass state-of-the-art neural network compression methods, establishing new benchmarks in both interpretability and efficiency for BNNs.
Technology Category
Application Category
📝 Abstract
Modeling natural phenomena with artificial neural networks (ANNs) often provides highly accurate predictions. However, ANNs often suffer from over-parameterization, complicating interpretation and raising uncertainty issues. Bayesian neural networks (BNNs) address the latter by representing weights as probability distributions, allowing for predictive uncertainty evaluation. Latent binary Bayesian neural networks (LBBNNs) further handle structural uncertainty and sparsify models by removing redundant weights. This article advances LBBNNs by enabling covariates to skip to any succeeding layer or be excluded, simplifying networks and clarifying input impacts on predictions. Ultimately, a linear model or even a constant can be found to be optimal for a specific problem at hand. Furthermore, the input-skip LBBNN approach reduces network density significantly compared to standard LBBNNs, achieving over 99% reduction for small networks and over 99.9% for larger ones, while still maintaining high predictive accuracy and uncertainty measurement. For example, on MNIST, we reached 97% accuracy and great calibration with just 935 weights, reaching state-of-the-art for compression of neural networks. Furthermore, the proposed method accurately identifies the true covariates and adjusts for system non-linearity. The main contribution is the introduction of active paths, enhancing directly designed global and local explanations within the LBBNN framework, that have theoretical guarantees and do not require post hoc external tools for explanations.
Problem
Research questions and friction points this paper is trying to address.
Addresses over-parameterization and uncertainty in neural networks.
Enhances interpretability by simplifying network structures.
Maintains high accuracy while significantly reducing network density.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Input-skip Latent Binary Bayesian Neural Networks
Reduces network density significantly
Enhances global and local explanations
🔎 Similar Papers
No similar papers found.
