Benchmarking Foundation Models for Mitotic Figure Classification

2025-08-06

AI Summary

Pathological image annotation scarcity severely limits mitosis classification performance, particularly undermining generalization across tumor types. To address this, we propose a lightweight adaptation framework built upon self-supervised foundation models: it employs a Vision Transformer (ViT) backbone and jointly leverages linear probing and Low-Rank Adaptation (LoRA) to optimize attention modules—enabling enhanced in-domain accuracy and out-of-domain robustness under limited labeling. Experiments demonstrate that LoRA fine-tuning with only 10% of the training data achieves 98% of the performance attained using the full dataset. Moreover, on cross-tumor domain generalization tasks, our method significantly narrows the performance gap, approaching that of fully fine-tuned CNNs and end-to-end ViTs. This work constitutes the first systematic validation of LoRA’s efficacy for improving cross-domain generalization of pathological foundation models. It establishes an efficient, practical paradigm for few-shot, multi-center histopathological analysis.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Large Vision Models
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The performance of deep learning models is known to scale with data quantity and diversity. In pathology, as in many other medical imaging domains, the availability of labeled images for a specific task is often limited. Self-supervised learning techniques have enabled the use of vast amounts of unlabeled data to train large-scale neural networks, i.e., foundation models, that can address the limited data problem by providing semantically rich feature vectors that can generalize well to new tasks with minimal training effort increasing model performance and robustness. In this work, we investigate the use of foundation models for mitotic figure classification. The mitotic count, which can be derived from this classification task, is an independent prognostic marker for specific tumors and part of certain tumor grading systems. In particular, we investigate the data scaling laws on multiple current foundation models and evaluate their robustness to unseen tumor domains. Next to the commonly used linear probing paradigm, we also adapt the models using low-rank adaptation (LoRA) of their attention mechanisms. We compare all models against end-to-end-trained baselines, both CNNs and Vision Transformers. Our results demonstrate that LoRA-adapted foundation models provide superior performance to those adapted with standard linear probing, reaching performance levels close to 100% data availability with only 10% of training data. Furthermore, LoRA-adaptation of the most recent foundation models almost closes the out-of-domain performance gap when evaluated on unseen tumor domains. However, full fine-tuning of traditional architectures still yields competitive performance.

Problem

• Address limited labeled data in mitotic figure classification
• Evaluate foundation models' robustness across unseen tumor domains
• Compare LoRA-adapted models with traditional fine-tuning methods

Innovation

• Self-supervised learning for foundation models
• Low-rank adaptation (LoRA) for attention mechanisms
• Robust out-of-domain performance evaluation