Don't waste SAM

2026-06-09

AI Summary

This study addresses the performance degradation of the general-purpose segmentation model SAM in real-world waste scenarios, where occlusion, deformation, transparency, and background clutter pose significant challenges. For the first time, the authors systematically evaluate and fine-tune SAM’s generalization capability across three waste datasets—Zerowaste, TACO, and TrashCan 1.0. Building upon the SAM-ViT-H architecture and employing a targeted supervised fine-tuning strategy, the approach substantially improves segmentation accuracy on complex waste objects. Experimental results demonstrate that the fine-tuned model achieves IoU gains exceeding 30 on both Zerowaste and TACO, and lags only 1.44 behind the state-of-the-art on TrashCan 1.0, significantly outperforming existing methods overall. These findings validate SAM’s effectiveness and potential as a foundational model for waste segmentation tasks.

Abstract

Meta AI has recently released the Segment Anything Model (SAM), which demonstrates exceptional zero-shot image segmentation performance across various tasks with remarkable accuracy. Despite its inability to provide accurate segmentation across multiple research fields, SAM still serves as a valuable starting point for supporting the segmentation pipeline process, particularly for tasks that require extensive and senior skills annotations. This study aims to evaluate the generalization of SAM and fine-tuning SAM models using three waste segmentation datasets. Although they are captured from real scenes as SAM was pretrained on, these datasets present several challenges, including occlusions, deformable objects, transparency, and objects easily confused with backgrounds. In our findings, the fine-tuned SAM-ViT-H model outperforms the state-ofthe-art Zerowaste, and TACO datasets with a significant increase of +30 in IoU, and it closely approaches performance levels of TrashCan 1.0, with only a -1.44 difference. After evaluating these popular waste datasets, it became evident that fine-tuning SAM as a foundational model is a crucial step for providing better generalization for downstream waste segmentation tasks. Therefore, SAM should not be disregarded or wasted.

Problem

• waste segmentation
• Segment Anything Model
• generalization
• occlusion
• transparent objects

Innovation

• Segment Anything Model
• fine-tuning
• waste segmentation
• zero-shot generalization
• foundation model