Vision Transformers for Kidney Stone Image Classification: A Comparative Study with CNNs

2025-08-18

AI Summary

Accurate classification of kidney stone endoscopic images is critical for personalized treatment planning and recurrence prevention; however, conventional convolutional neural networks (CNNs) struggle to model long-range spatial dependencies and exhibit limited performance under multi-modal imaging conditions. To address this, we conduct the first systematic evaluation of Vision Transformers (ViTs) for fine-grained kidney stone classification. Specifically, we compare ImageNet-21k pre-trained ViT-base against ResNet-50 on a dual-source ex vivo dataset acquired via CCD cameras and flexible ureteroscopes. Results demonstrate that ViT achieves 95.2% accuracy and 95.1% F1-score on the most challenging subset—substantially outperforming ResNet-50 (64.5% accuracy, 59.3% F1). Moreover, ViT exhibits superior robustness and generalizability across diverse imaging scenarios. This work establishes ViT as a highly effective architecture for medical endoscopic image classification, offering a novel paradigm for intelligent diagnosis in minimally invasive urological surgery.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Kidney stone classification from endoscopic images is critical for personalized treatment and recurrence prevention. While convolutional neural networks (CNNs) have shown promise in this task, their limited ability to capture long-range dependencies can hinder performance under variable imaging conditions. This study presents a comparative analysis between Vision Transformers (ViTs) and CNN-based models, evaluating their performance on two ex vivo datasets comprising CCD camera and flexible ureteroscope images. The ViT-base model pretrained on ImageNet-21k consistently outperformed a ResNet50 baseline across multiple imaging conditions. For instance, in the most visually complex subset (Section patches from endoscopic images), the ViT model achieved 95.2% accuracy and 95.1% F1-score, compared to 64.5% and 59.3% with ResNet50. In the mixed-view subset from CCD-camera images, ViT reached 87.1% accuracy versus 78.4% with CNN. These improvements extend across precision and recall as well. The results demonstrate that ViT-based architectures provide superior classification performance and offer a scalable alternative to conventional CNNs for kidney stone image analysis.

Problem

• Classifying kidney stones from endoscopic images
• Comparing Vision Transformers with CNN performance
• Improving classification accuracy under variable imaging conditions

Innovation

• Vision Transformers outperform CNNs classification
• ViT-base model achieves higher accuracy F1-score
• Superior performance across multiple imaging conditions