Neighborhood Smoothing for Calibration

2026-10-06

AI Summary

This study addresses the overconfidence of neural networks and the limitation that existing training-time calibration methods overlook the neighborhood structure in representation spaces. To this end, this work proposes a graph smoothing-based calibration regularization framework. Specifically, it constructs a nearest-neighbor graph within the learned representation space and regularizes the model by penalizing the Jensen-Shannon divergence between the predictive distributions of adjacent samples. Furthermore, this paper is the first to establish graph smoothing as a general principle, theoretically deriving bounds relating neighborhood prediction discrepancies to local confidence variations. Experimental results demonstrate that the proposed method significantly improves predictive quality. When combined with temperature scaling, it achieves the lowest negative log-likelihood on most benchmarks, outperforming existing training-time calibration approaches.

Abstract

Modern neural networks are often miscalibrated, with a tendency to overconfidence. Existing train-time calibration methods largely modify task losses or calibration penalties, leaving neighborhood structure in learned representations underexploited. We introduce graph smoothing as a general principle for train-time calibration, which encourages similar predictive distributions across neighboring samples in representation space. We analyze the effects of graph smoothing, deriving bounds that connect predictive divergence between neighboring samples to local confidence variation and to the propagation of pointwise calibration error, and characterize the conditions under which smoothing can or cannot improve calibration. In light of this analysis, we propose \modelNoSpace, a graph-based train-time regularizer that penalizes the Jensen--Shannon divergence between predictive distributions of neighboring samples. We present a thorough empirical analysis, showing that across standard calibration benchmarks, \model improves predictive quality, and the improvement is complementary to post-hoc calibration: after temperature scaling, \model attains the lowest NLL of all evaluated train-time methods in seven of the eight image and tabular settings. These findings demonstrate the value of graph smoothing over learned representations for neural network calibration.

Problem

• neural network calibration
• overconfidence
• train-time calibration
• representation space
• neighborhood structure

Innovation

• Neighborhood Smoothing
• Graph Smoothing
• Train-time Calibration
• Jensen-Shannon Divergence
• Representation Space