Beyond Classification: Pathology Foundation Models as Detection Encoders for Mitotic Figures

2026-07-30

AI Summary

This study presents the first systematic evaluation of foundation models in pathology for the task of mitotic cell detection—a dense object detection problem—moving beyond their conventional use in image classification. Leveraging self-supervised pretrained encoders such as UNI, Virchow, and H-optimus-0 within established detection frameworks including RetinaNet, Faster R-CNN, and Deformable DETR, the authors conduct experiments on the multi-domain MIDOG++ and TUPAC16 datasets. The results demonstrate that H-optimus-0 and Virchow achieve performance comparable to end-to-end trained baselines while exhibiting superior robustness on out-of-domain data, thereby confirming the potential and generalization capability of image-level self-supervised foundation models in dense detection tasks.

Abstract

Pathology foundation models (FMs) are models trained on vast amounts of typically unlabeled data and have been shown to yield regularized latent spaces that can be used effectively in downstream classification tasks. This is also true for the classification of mitotic figures vs. other cells. However, it is so far unclear if the latent space of current FMs provides features that are discriminant and spatially suitably resolved to also serve as a backbone for dense object detection paradigms. In this work, we investigate this question for common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1) and compare their performance against a fully end-to-end trained baseline based on a ResNet50 architecture. We combine FM backbones with representatives of single stage, dual stage and self-attention-based detectors (RetinaNet, Faster R-CNN, Deformable DETR respectively) on the multi-domain MIDOG++ dataset, and on the TUPAC16 dataset as an out-of-domain case. We show that the H-optimus-0 and Virchow models yielded competitive performance, indicating that the latent spaces of current FMs, all trained on image-level self-supervision, are suitable for direct mitotic figure detection and may be slightly more robust on our out-of-domain test case. All code is made available publicly at https://github.com/DeepMicroscopy/FM4MFdet.

Problem

• pathology foundation models
• mitotic figure detection
• dense object detection
• latent space
• out-of-domain generalization

Innovation

• Pathology Foundation Models
• Mitotic Figure Detection
• Dense Object Detection
• Self-supervised Learning
• Cross-domain Robustness