Technology Category
Machine Learning: Calibration & Uncertainty QuantificationReasoning under Uncertainty: Uncertainty RepresentationsNatural Language Processing: Safety and Robustness
Similarity-Distance-Magnitude Universal Verification
🤖 AI Summary
This work addresses two fundamental challenges in deep learning: weak neural network robustness and the lack of human-interpretable uncertainty quantification. To this end, we propose the Similarity-Distance-Magnitude (SDM) activation function, introducing the first triple-aware mechanism that jointly perceives prediction similarity, distance to the training distribution, and output magnitude. Methodologically, we integrate class-conditional empirical cumulative distribution function (CDF) modeling, probabilistic region-wise calibration, and uncertainty-driven selective computation—including LLM routing and multi-model consensus verification—to enable conditional uncertainty estimation, class-specific accuracy assessment, and robust selective inference under distributional shift. Experiments demonstrate substantial improvements in out-of-distribution (OOD) detection and OOD robustness. The approach yields interpretable uncertainty summaries—e.g., calibrated confidence intervals and credibility quantiles—and supports selective classification, conditional generation, and model consensus validation. Overall, it establishes a new paradigm for trustworthy AI that balances theoretical rigor with practical deployability.
Technology Category
Application Category
📝 Abstract
We solve the neural network robustness problem by adding Similarity (i.e., correctly predicted depth-matches into training)-awareness and Distance-to-training-distribution-awareness to the existing output Magnitude (i.e., decision-boundary)-awareness of the softmax function. The resulting sdm activation function provides strong signals of the relative epistemic (reducible) predictive uncertainty. We use this novel behavior to further address the complementary HCI problem of mapping the output to human-interpretable summary statistics over relevant partitions of a held-out calibration set. Estimates of prediction-conditional uncertainty are obtained via a parsimonious learned transform over the class-conditional empirical CDFs of the output of a final-layer sdm activation function. For decision-making and as an intrinsic model check, estimates of class-conditional accuracy are obtained by further partitioning the high-probability regions of this calibrated output into class-conditional, region-specific CDFs. The uncertainty estimates from sdm calibration are remarkably robust to test-time distribution shifts and out-of-distribution inputs; incorporate awareness of the effective sample size; provide estimates of uncertainty from the learning and data splitting processes; and are well-suited for selective classification and conditional branching for additional test-time compute based on the predictive uncertainty, as for selective LLM generation, routing, and composition over multiple models and retrieval. Finally, we construct sdm networks, LLMs with uncertainty-aware verification and interpretability-by-exemplar as intrinsic properties. We provide open-source software implementing these results.
Problem
Research questions and friction points this paper is trying to address.
Enhance neural network robustness via novel activation function.
Map model outputs to human-interpretable summary statistics.
Ensure uncertainty estimates robust to distribution shifts.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Similarity-Distance-Magnitude activation
epistemic predictive uncertainty
uncertainty-aware LLMs verification
🔎 Similar Papers
No similar papers found.
