🤖 AI Summary
Precise segmentation of orthopedic anatomical landmarks—particularly fine-grained fiducial points in pelvic X-ray images—remains challenging: general foundation models like SAM lack orthopedic priors, while MedSAM targets only coarse-grained organ-level structures, failing to achieve the sub-millimeter accuracy required for clinical diagnosis. To address this, we propose YOLO-SAM, the first hybrid framework integrating YOLOv8 object detection with SAM-based segmentation. YOLOv8 generates high-confidence bounding boxes that serve as spatial prompts for SAM, enabling accurate delineation of 72 critical landmarks and complex bony structures. This two-stage paradigm synergistically combines robust localization with shape-preserving segmentation. Evaluated on pelvic X-ray data, YOLO-SAM significantly improves segmentation accuracy for small and irregularly shaped targets (mAP ↑12.6%, Dice ↑9.3%). The framework establishes an interpretable, deployable paradigm for landmark-driven quantitative orthopedic analysis.
📝 Abstract
Radiographic images are a cornerstone of medical diagnostics in orthopaedics, with anatomical landmark detection serving as a crucial intermediate step for information extraction. General-purpose foundational segmentation models, such as SAM (Segment Anything Model), do not support landmark segmentation out of the box and require prompts to function. However, in medical imaging, the prompts for landmarks are highly specific. Since SAM has not been trained to recognize such landmarks, it cannot generate accurate landmark segmentations for diagnostic purposes. Even MedSAM, a medically adapted variant of SAM, has been trained to identify larger anatomical structures, such as organs and their parts, and lacks the fine-grained precision required for orthopaedic pelvic landmarks. To address this limitation, we propose leveraging another general-purpose, non-foundational model: YOLO. YOLO excels in object detection and can provide bounding boxes that serve as input prompts for SAM. While YOLO is efficient at detection, it is significantly outperformed by SAM in segmenting complex structures. In combination, these two models form a reliable pipeline capable of segmenting not only a small pilot set of eight anatomical landmarks but also an expanded set of 72 landmarks and 16 regions with complex outlines, such as the femoral cortical bone and the pelvic inlet. By using YOLO-generated bounding boxes to guide SAM, we trained the hybrid model to accurately segment orthopaedic pelvic radiographs. Our results show that the proposed combination of YOLO and SAM yields excellent performance in detecting anatomical landmarks and intricate outlines in orthopaedic pelvic radiographs.