Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning

2025-09-12

AI Summary

Medical vision foundation models (Med-VFMs) face challenges in efficient source-free domain adaptation (SFDA) for volumetric medical image segmentation, where source-domain data are inaccessible and target-domain annotation budgets are severely limited. Method: We propose an active SFDA framework that jointly quantifies target-sample informativeness via two metrics—knowledge discrepancy and anatomical segmentation difficulty—and incorporates a reliability discrimination mechanism to select high-value samples. Further, we introduce a self-supervised prior-guided selective semi-supervised fine-tuning strategy, enabling dynamic test-time querying and efficient optimization without source data. Contribution/Results: Our method significantly improves cross-domain segmentation performance under constrained labeling budgets. Evaluated on multiple 3D medical segmentation benchmarks, it outperforms both random sampling and conventional active learning approaches, achieving superior adaptation accuracy with minimal annotation cost.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Active Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising

Abstract

Medical Vision Foundation Models (Med-VFMs) have superior capabilities of interpreting medical images due to the knowledge learned from self-supervised pre-training with extensive unannotated images. To improve their performance on adaptive downstream evaluations, especially segmentation, a few samples from target domains are selected randomly for fine-tuning them. However, there lacks works to explore the way of adapting Med-VFMs to achieve the optimal performance on target domains efficiently. Thus, it is highly demanded to design an efficient way of fine-tuning Med-VFMs by selecting informative samples to maximize their adaptation performance on target domains. To achieve this, we propose an Active Source-Free Domain Adaptation (ASFDA) method to efficiently adapt Med-VFMs to target domains for volumetric medical image segmentation. This ASFDA employs a novel Active Learning (AL) method to select the most informative samples from target domains for fine-tuning Med-VFMs without the access to source pre-training samples, thus maximizing their performance with the minimal selection budget. In this AL method, we design an Active Test Time Sample Query strategy to select samples from the target domains via two query metrics, including Diversified Knowledge Divergence (DKD) and Anatomical Segmentation Difficulty (ASD). DKD is designed to measure the source-target knowledge gap and intra-domain diversity. It utilizes the knowledge of pre-training to guide the querying of source-dissimilar and semantic-diverse samples from the target domains. ASD is designed to evaluate the difficulty in segmentation of anatomical structures by measuring predictive entropy from foreground regions adaptively. Additionally, our ASFDA method employs a Selective Semi-supervised Fine-tuning to improve the performance and efficiency of fine-tuning by identifying samples with high reliability from unqueried ones.

Problem

• Adapting medical foundation models efficiently to target domains
• Selecting informative samples via active learning for fine-tuning
• Improving volumetric medical image segmentation with minimal annotation

Innovation

• Active Learning for sample selection
• Diversified Knowledge Divergence metric
• Selective Semi-supervised Fine-tuning approach