HMS-VesselNet: Hierarchical Multi-Scale Attention Network with Topology-Preserving Loss for Retinal Vessel Segmentation

2026-03-23

AI Summary

This work addresses the challenge of accurately segmenting low-contrast, small-scale peripheral retinal vessels, which existing methods struggle to detect due to their reliance on standard overlap-based losses. To overcome this limitation, we propose a hierarchical multi-scale attention network with four parallel branches that learns to fuse features across multiple resolutions through trainable weights. The model is jointly optimized using a composite loss function combining Dice, binary cross-entropy, and centerline Dice to balance region coverage and topological continuity. Furthermore, hard example mining is incorporated to enhance robustness. Evaluated on DRIVE, STARE, and CHASE_DB1 datasets, our method achieves an average Dice score of 88.72%, sensitivity of 90.78%, and AUC of 98.25%, with cross-dataset AUC consistently exceeding 95%. Notably, it demonstrates significantly improved recall for fine vessels compared to current state-of-the-art approaches.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Retinal vessel segmentation methods based on standard overlap losses tend to miss thin peripheral vessels because these structures occupy very few pixels and have low contrast against the background. We propose HMS-VesselNet, a hierarchical multi-scale network that processes fundus images across four parallel branches at different resolutions and combines their outputs using learned fusion weights. The training loss combines Dice, binary cross-entropy, and centerline Dice to jointly optimize area overlap and vessel continuity. Hard example mining is applied from epoch 20 onward to concentrate gradient updates on the most difficult training images. Tested on 68 images from DRIVE, STARE, and CHASE_DB1 using 5-fold cross-validation, the model achieves a mean Dice of 88.72 +/- 0.67%, Sensitivity of 90.78 +/- 1.42%, and AUC of 98.25 +/- 0.21%. In leave-one-dataset-out experiments, AUC remains above 95% on each unseen dataset. The largest improvement is in the recall of thin peripheral vessels, which are the structures most frequently missed by standard methods and most critical for early detection of diabetic retinopathy.

Problem

• retinal vessel segmentation
• thin peripheral vessels
• low contrast
• standard overlap losses
• diabetic retinopathy

Innovation

• hierarchical multi-scale attention
• topology-preserving loss
• centerline Dice
• hard example mining
• retinal vessel segmentation