🤖 AI Summary
Traditional neural networks struggle to provide reliable uncertainty estimates, while Bayesian neural networks, despite their theoretical advantages, are computationally expensive and difficult to scale. This work proposes a novel approach that integrates Dirichlet distributions with Monte Carlo Dropout to structurally model predictive class probabilities, thereby yielding better-calibrated and more informative uncertainty representations while retaining the computational efficiency of Monte Carlo Dropout during inference. The method demonstrates significant improvements over existing techniques across multiple benchmark tasks, offering a practical pathway toward deploying efficient and reliable uncertainty-aware deep models in real-world applications.
📝 Abstract
Traditional neural networks provide deterministic predictions without inherent uncertainty estimates. While Bayesian Neural Networks (BNNs) offer a principled approach to uncertainty quantification, their computational complexity limits scalability. Monte Carlo (MC) Dropout, initially introduced as a regularization technique, has been shown to approximate Bayesian inference by enabling probabilistic modeling through multiple stochastic forward passes. In this work, we enhance uncertainty estimation in deep learning by integrating a Dirichlet-based framework within MC Dropout. Specifically, we leverage the formulation proposed by Sensoy et al. (2018), where class probabilities are modeled using a Dirichlet distribution, allowing for a more informative uncertainty representation. The proposed approach maintains the computational efficiency of MC Dropout while improving the quality of uncertainty estimates. We discuss the theoretical foundations of our method and compare it with existing uncertainty quantification techniques. The results highlight the effectiveness of the proposed method in producing well-calibrated uncertainty estimates, offering a practical solution for uncertainty-aware deep learning models.