Prompt-Free SAM-Based Multi-Task Framework for Breast Ultrasound Lesion Segmentation and Classification

2026-01-09

arXiv.org

AI Summary

Breast ultrasound image analysis remains highly challenging due to low contrast, speckle noise, and diverse lesion morphologies, which complicate tumor segmentation and classification. This work proposes a novel multi-task deep learning framework that, for the first time, dispenses with conventional prompting mechanisms and instead leverages the prompt-free high-dimensional features from the Segment Anything Model (SAM) vision encoder. These features are combined with a lightweight convolutional or U-Net-style decoder for fully supervised segmentation, while a mask-guided attention mechanism is introduced to enhance the classification branch, enabling synergistic optimization of segmentation and diagnostic tasks. Evaluated on the PRECISE 2025 dataset, the proposed method achieves a Dice coefficient of 0.887 and a classification accuracy of 92.3%, ranking among the top performers in the challenge.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Accurate tumor segmentation and classification in breast ultrasound (BUS) imaging remain challenging due to low contrast, speckle noise, and diverse lesion morphology. This study presents a multi-task deep learning framework that jointly performs lesion segmentation and diagnostic classification using embeddings from the Segment Anything Model (SAM) vision encoder. Unlike prompt-based SAM variants, our approach employs a prompt-free, fully supervised adaptation where high-dimensional SAM features are decoded through either a lightweight convolutional head or a UNet-inspired decoder for pixel-wise segmentation. The classification branch is enhanced via mask-guided attention, allowing the model to focus on lesion-relevant features while suppressing background artifacts. Experiments on the PRECISE 2025 breast ultrasound dataset, split per class into 80 percent training and 20 percent testing, show that the proposed method achieves a Dice Similarity Coefficient (DSC) of 0.887 and an accuracy of 92.3 percent, ranking among the top entries on the PRECISE challenge leaderboard. These results demonstrate that SAM-based representations, when coupled with segmentation-guided learning, significantly improve both lesion delineation and diagnostic prediction in breast ultrasound imaging.

Problem

• breast ultrasound
• lesion segmentation
• lesion classification
• speckle noise
• low contrast

Innovation

• Prompt-Free
• SAM-Based
• Multi-Task Learning
• Mask-Guided Attention
• Breast Ultrasound Segmentation