Ranking pre-trained segmentation models for zero-shot transferability

2025-03-01

AI Summary

In zero-shot transfer for microscopy image segmentation, selecting appropriate pre-trained models is challenging due to the absence of target labels and source training data. Method: We propose the first fully unsupervised, source-data-free, and label-free transferability estimation method, grounded in generalization theory and the perturbation consistency assumption. Our framework quantifies output stability under input perturbations, integrating feature-space consistency and prediction confidence calibration. Contribution/Results: This is the first approach enabling truly zero-shot, source-independent transferability assessment for both semantic and instance segmentation. Evaluated on multimodal microscopy datasets, it achieves high correlation (Spearman ρ > 0.85) between estimated model rankings and actual target-domain performance—significantly outperforming existing unsupervised baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Model transfer presents a solution to the challenges of segmentation in the microscopy community, where the immense cost of labelling sufficient training data is a major bottleneck in the use of deep learning. With large quantities of imaging data produced across a wide range of imaging conditions, institutes also produce many bespoke models trained on specific source data which then get collected in model banks or zoos. As the number of available models grows, so does the need for an efficient and reliable model selection method for a specific target dataset of interest. We focus on the unsupervised regime where no labels are available for the target dataset. Building on previous work linking model generalisation and consistency under perturbation, we propose the first unsupervised transferability estimator for semantic and instance segmentation tasks which doesn't require access to source training data or target domain labels. We evaluate the method on multiple segmentation problems across microscopy modalities, finding a strong correlation between the rankings based on our estimator and rankings based on target dataset performance.

Problem

• Ranking pre-trained models for zero-shot transferability in microscopy segmentation.
• Developing unsupervised transferability estimator without source data or target labels.
• Evaluating method on microscopy modalities for semantic and instance segmentation.

Innovation

• Unsupervised transferability estimator for segmentation tasks
• No need for source data or target labels
• Strong correlation with target dataset performance