Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening

2026-05-25

AI Summary

This study addresses the lack of systematic model comparisons in multi-disease retinal screening by conducting the first comprehensive evaluation of twelve prominent vision models—including CNNs, Vision Transformers (ViTs), hybrid architectures, and vision-language models such as CLIP and SigLIP—under a unified training, calibration, and evaluation protocol. The assessment encompasses both binary classification and 28-class multilabel tasks on the RFMiD dataset, with external validation on Messidor-2. Results show that all models achieve internal test AUCs exceeding 84%, with Swin-Tiny, CoAtNet-0, and MaxViT-Tiny performing best. On external validation, AUCs range from 66.8% to 84.7%, with hybrid and Transformer-based architectures generally outperforming others. This work establishes a reproducible performance benchmark and provides evidence-based guidance for model selection at clinically relevant operating points.

Abstract

Modern deep learning offers powerful tools for automated retinal screening, but it remains unclear how different visual model families compare in realistic multi-disease settings and under domain shift. In this work, we benchmark twelve architectures across four model families: convolutional neural networks, vision transformers, hybrid CNN-transformer backbones, and vision-language models, using the Retinal Fundus Multi-disease Image Dataset (RFMiD). We evaluate two tasks: binary screening for any retinal disease and multi-label classification across 28 disease classes. Using standardized training, calibration, and evaluation protocols, we report AUC, F1, precision, recall, and sensitivity at a clinically relevant operating point with specificity near 80%. On RFMiD, all architectures perform well on binary screening, with AUC above 84%, but attention-based models perform best. SwinTiny and the hybrid CoAtNet0 and MaxViTTiny models achieve the strongest binary screening results and improve macro and micro F1 in the multi-label setting. Vision-language models, including CLIP ViT-B/16 and SigLIP-Base384, are competitive with CNN baselines but do not surpass the best transformer and hybrid backbones. In external validation on Messidor-2 for referable diabetic retinopathy, AUC ranges from 66.8% to 84.7%, with hybrid and transformer models again showing strong performance. These results provide a reproducible reference for model selection in multi-disease retinal screening and guide future automated screening tools for clinical deployment.

Problem

• retinal screening
• multi-disease classification
• domain shift
• model benchmarking
• deep learning

Innovation

• retinal screening
• vision transformers
• hybrid models
• vision-language models
• multi-disease classification