Rethinking Few-Shot Medical Image Segmentation by SAM2: A Training-Free Framework with Augmentative Prompting and Dynamic Matching

2025-03-05

AI Summary

Medical image segmentation heavily relies on large-scale annotated datasets, yet existing few-shot methods still require substantial training samples, hindering efficient clinical deployment. To address this, we propose a training-free few-shot 3D medical image segmentation framework: 3D volumetric data is reformulated as a video sequence, leveraging SAM2’s temporal modeling capability; only a single support image is needed—through data augmentation and frame-level dynamic matching, mask prompts are generated to directly drive SAM2 for query volume segmentation. Key contributions include: (i) the first training-free paradigm that eliminates fine-tuning; (ii) a support-query frame similarity-driven dynamic matching strategy; and (iii) the first reformulation of 3D medical segmentation as a video segmentation task. Our method achieves state-of-the-art performance on mainstream few-shot benchmarks, significantly improving both segmentation accuracy and annotation efficiency, while offering plug-and-play clinical applicability.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multimodal Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Data transparency and provenance

Abstract

The reliance on large labeled datasets presents a significant challenge in medical image segmentation. Few-shot learning offers a potential solution, but existing methods often still require substantial training data. This paper proposes a novel approach that leverages the Segment Anything Model 2 (SAM2), a vision foundation model with strong video segmentation capabilities. We conceptualize 3D medical image volumes as video sequences, departing from the traditional slice-by-slice paradigm. Our core innovation is a support-query matching strategy: we perform extensive data augmentation on a single labeled support image and, for each frame in the query volume, algorithmically select the most analogous augmented support image. This selected image, along with its corresponding mask, is used as a mask prompt, driving SAM2's video segmentation. This approach entirely avoids model retraining or parameter updates. We demonstrate state-of-the-art performance on benchmark few-shot medical image segmentation datasets, achieving significant improvements in accuracy and annotation efficiency. This plug-and-play method offers a powerful and generalizable solution for 3D medical image segmentation.

Problem

• Reduces reliance on large labeled datasets for medical image segmentation.
• Introduces a training-free framework using SAM2 for few-shot learning.
• Enhances 3D medical image segmentation accuracy and annotation efficiency.

Innovation

• Uses SAM2 for video-like 3D medical segmentation
• Implements augmentative prompting with dynamic matching
• Eliminates need for model retraining or updates