Zero-shot 3D Segmentation of Abdominal Organs in CT Scans Using Segment Anything Model 2: Adapting Video Tracking Capabilities for 3D Medical Imaging

2024-08-12

arXiv.org

AI Summary

This study investigates the zero-shot 3D organ segmentation capability of SAM 2 on abdominal CT volumes, focusing on its generalization to multiple organs (e.g., spleen, kidneys) without fine-tuning. Method: We pioneer the adaptation of SAM 2’s video temporal modeling mechanism to medical volumetric data, proposing a zero-shot 3D adaptation framework that leverages z-axis–based cross-slice prompt initialization and negative prompting to guide exclusion of irrelevant regions in voxel space. Contribution/Results: Our approach achieves high segmentation accuracy for large organs (Dice scores: spleen 0.891, left kidney 0.870) and significantly improves Dice scores across six organs upon incorporating negative prompts—validating their critical regulatory role. This work establishes a reproducible, annotation-free, and fine-tuning–free paradigm for general-purpose medical image segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

Objectives: To evaluate the zero-shot performance of Segment Anything Model 2 (SAM 2) in 3D segmentation of abdominal organs in CT scans, and to investigate the effects of prompt settings on segmentation results. Materials and Methods: In this retrospective study, we used a subset of the TotalSegmentator CT dataset from eight institutions to assess SAM 2's ability to segment eight abdominal organs. Segmentation was initiated from three different z-coordinate levels (caudal, mid, and cranial levels) of each organ. Performance was measured using the Dice similarity coefficient (DSC). We also analyzed the impact of"negative prompts,"which explicitly exclude certain regions from the segmentation process, on accuracy. Results: 123 patients (mean age, 60.7 pm 15.5 years; 63 men, 60 women) were evaluated. As a zero-shot approach, larger organs with clear boundaries demonstrated high segmentation performance, with mean DSCs as follows: liver 0.821 pm 0.192, right kidney 0.862 pm 0.212, left kidney 0.870 pm 0.154, and spleen 0.891 pm 0.131. Smaller organs showed lower performance: gallbladder 0.531 pm 0.291, pancreas 0.361 pm 0.197, and adrenal glands, right 0.203 pm 0.222, left 0.308 pm 0.234. The initial slice for segmentation and the use of negative prompts significantly influenced the results. By removing negative prompts from the input, the DSCs significantly decreased for six organs. Conclusion: SAM 2 demonstrated promising zero-shot performance in segmenting certain abdominal organs in CT scans, particularly larger organs. Performance was significantly influenced by input negative prompts and initial slice selection, highlighting the importance of optimizing these factors.

Problem

• Segment Anything Model 2
• CT Image Segmentation
• Abdominal Organ

Innovation

• Segment Anything Model 2
• Automatic Organ Segmentation
• CT Images