A Unified 2D Framework for DeepLesion Detection, Segmentation and Short Report Generation

2026-08-03

AI Summary

This work addresses the challenges of lesion segmentation in the DeepLesion dataset and the absence of a unified framework for simultaneous detection, segmentation, and radiology report generation. To this end, we propose the first end-to-end 2D lesion analysis framework that jointly optimizes bounding box detection, pixel-level segmentation, and short radiology report generation powered by a large language model. By effectively integrating spatial and anatomical contextual information, our method achieves substantial improvements across all tasks: on the original DeepLesion dataset, it attains an mAP50 of 70.1%, a Dice score of 62.6%—a 28.5% relative improvement over nnUNet—and a BLEU_1 score of 64.3%. The code and trained models are publicly released.

Abstract

In previous work, we integrated large language models (LLMs) into the lesion segmentation model based on the ULS23 DeepLesion dataset, using short-form findings from the reports. In this study, we developed a unified 2D lesion analysis framework that integrates LLM-based reasoning, lesion bounding box detection, segmentation, and radiology report generation from the original DeepLesion dataset. In the testing phase, we achieved relatively high lesion bounding box detection accuracy with mAP50 of 70.1%, mAP50-95 of 46.4%; Lesion segmentation performance with a Dice score of 62.6%; short report generation accuracy with BLEU_1 score of 64.3%, BLEU_4 score of 49.6%, METEOR of 34.7%, and ROUGE_L of 60.1%. In this work, we address the challenging issue of segmentation in the original DeepLesion dataset and achieve a 28.5% Dice score improvement over the nnUNet lesion segmentation model. We also integrated spatial and anatomical context into the DeepLesion short report generation. We released the implementation, dataset, and models on Github. https://github.com/ruida/2D_DeepLesion_Foundation

Problem

• lesion segmentation
• DeepLesion dataset
• report generation
• bounding box detection
• medical image analysis

Innovation

• unified 2D framework
• lesion segmentation
• large language models
• radiology report generation
• DeepLesion dataset