XFlow: A Workflow Model for Instruction-Guided Lesion Segmentation in Chest X-rays

2026-09-28

AI Summary

This study addresses the limitations of existing medical segmentation models, including narrow disease coverage, heavy reliance on lesion priors, and insufficient single-step prediction accuracy, by proposing an instruction-guided framework for chest X-ray lesion segmentation. Inspired by the coarse-to-fine visual perception mechanism of radiologists, the method introduces a multi-stage workflow to explicitly model the decision-making process. Technically, it builds upon a fine-tuned Segment Anything Model (SAM) architecture, integrating bounding box localization with a multi-round interactive point prompt refinement strategy to effectively overcome the constraints of single-pass inference. Experimental results demonstrate that the proposed approach achieves state-of-the-art segmentation performance in both internal and external evaluations, significantly outperforming the ROSALIA baseline.

Abstract

Existing text-guided segmentation models in the medical domain cover only a narrow set of anatomical structures and lesions in chest X-rays (CXRs), and most of them assume that the queried target is always present in the image. Instruction-guided lesion segmentation (ILS) was introduced to overcome these limitations by segmenting diverse lesion types from simple user instructions while also recognizing when the queried lesion is absent, and ROSALIA was proposed as the first model for this task. However, the masks produced by ROSALIA remain of limited quality, often carrying scattered noise. Moreover, ROSALIA predicts the mask in a single shot, which differs fundamentally from how radiologists perceive and delineate lesions in practice. A radiologist first surveys the entire thorax, then localizes the approximate region of abnormality, and only then refines the lesion contour. Motivated by this coarse-to-fine, multi-level perception process, we present XFlow, a workflow model for ILS that combines box-based localization with multi-turn point refinement. XFlow detects the lungs, decides whether the queried finding is present in each of them, and prompts a fine-tuned SAM with the lesion box for an initial mask. It then corrects that mask through point prompts until its boundary follows the lesion, leaving every intermediate decision visible. Our experiments show that XFlow achieves the best segmentation quality on both internal and external evaluation. Notably, it surpasses ROSALIA in segmentation quality even when the two are trained on the same lesion annotations. Code and model weights will be made publicly available.

Problem

• Instruction-guided lesion segmentation
• Chest X-rays
• Coarse-to-fine perception
• Lesion segmentation quality

Innovation

• Instruction-Guided Lesion Segmentation
• Workflow Model
• Coarse-to-Fine Perception
• Segment Anything Model (SAM)
• Point Prompt Refinement