Beyond scalar losses: calibrating segmentation models via gradient vector field surgery

2026-07-15

AI Summary

Medical image segmentation models trained with region-based losses, such as Dice loss, commonly suffer from poor calibration and overconfident predictions, hindering their deployment in high-stakes clinical settings. This work is the first to uncover the root cause of this issue from a gradient perspective and introduces a novel “gradient vector field surgery” approach. By incorporating a scaling factor—linearly dependent on prediction error—into the partial derivatives of the loss function, the method enhances model calibration without altering the original loss structure. The proposed technique is highly generalizable and consistently improves calibration across diverse 2D and 3D medical segmentation tasks while preserving state-of-the-art segmentation accuracy.

Abstract

Region-based loss functions, such as the Dice loss, have established themselves as the de facto standard for highly class- and region-imbalanced segmentation tasks. However, models trained using region-based loss functions are notoriously miscalibrated and typically yield over-confident predictions. In medical imaging applications, such as defining tumor resection margins, this miscalibration is hindering clinical adoption. In this work, we outline a novel gradient perspective on this overconfidence and show how it affects region-based loss functions. We propose a "surgery" on the gradient vector field as a simple, yet effective intervention to mitigate calibration issues. This surgery adds a factor to the loss's partial derivative, scaling the gradient's magnitude linearly with the prediction error. In empirical evaluations across 2D and 3D medical segmentation tasks, we demonstrate the effectiveness of this intervention while maintaining high prediction accuracy when used in conjunction with any region-based loss function.

Problem

• segmentation calibration
• region-based loss
• overconfidence
• medical image segmentation
• model miscalibration

Innovation

• gradient vector field surgery
• model calibration
• region-based loss
• medical image segmentation
• overconfidence mitigation