Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study

2025-03-28

AI Summary

Domain shift arising from variations in imaging modalities and acquisition protocols severely hampers generalization in medical image segmentation. Method: This work presents the first systematic evaluation of six 3D medical foundation models (e.g., MedSAM, nnUNeXt) on zero-shot domain generalization across 12 publicly available cross-modality and cross-anatomy datasets. We propose an intelligent prompt-learning framework for zero-shot transfer, integrating anatomical prior injection and cross-domain feature alignment, and employ multimodal consistency analysis to elucidate generalization mechanisms. Contribution/Results: Our approach achieves up to a 12.7% Dice score improvement on unseen domains—approaching supervised fine-tuning performance. The study demonstrates that promptable foundation models possess substantial zero-shot domain generalization capability; critically, prompt design, anatomical prior guidance, and feature alignment constitute key levers for performance gain. This work establishes both a methodological foundation and an empirical benchmark for zero-shot generalization in 3D medical imaging.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Diffusion Models for Vision
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Domain shift, caused by variations in imaging modalities and acquisition protocols, limits model generalization in medical image segmentation. While foundation models (FMs) trained on diverse large-scale data hold promise for zero-shot generalization, their application to volumetric medical data remains underexplored. In this study, we examine their ability towards domain generalization (DG), by conducting a comprehensive experimental study encompassing 6 medical segmentation FMs and 12 public datasets spanning multiple modalities and anatomies. Our findings reveal the potential of promptable FMs in bridging the domain gap via smart prompting techniques. Additionally, by probing into multiple facets of zero-shot DG, we offer valuable insights into the viability of FMs for DG and identify promising avenues for future research.

Problem

• Addressing domain shift in 3D medical image segmentation
• Evaluating foundational models for zero-shot generalization
• Exploring prompt techniques to bridge domain gaps

Innovation

• Utilizes promptable foundational models for segmentation
• Employs smart prompting to bridge domain gaps
• Tests zero-shot generalization across diverse datasets